{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ffe85c98-30f4-474b-8b92-dae4c0785664",
   "metadata": {},
   "source": [
    "https://langchain-ai.github.io/langgraph/how-tos/react-agent-structured-output/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ed7c34-36c0-4fbb-8919-cc53b9ee4d98",
   "metadata": {},
   "source": [
    "- model & application：密不可分，都十分必要\n",
    "- `LangChain`/`LangGraph`：提供了很多脚手架和工具，适当上手之后，会极大的简化开发；\n",
    "    - 虽然目前我只选择这两个工具，很多设计是可以复用的，可以很快地切到其他的 LLM dev framework；\n",
    "    - 入门和上手：不断的消化基本概念、基本设计，我觉得是非常必要的，相当大的比例是跟 openai api 对齐的（function calling）以及最新的 llm 的科研论文；\n",
    "- Structured output：而非自然语言，而希望达到 100%，出于自动化的目的；\n",
    "    - 基础是 function calling，response model as a tool\n",
    "    - llm 结果评估，整个 workflow/pipeline 中间环节的一部分\n",
    "        - 避免很多字符串繁琐的基于正则的解析\n",
    "    - llm 合成数据\n",
    "    - 基于 llm 对原始的非结构化数据做结构化的提取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f1f5f370-432d-451d-92da-dd3adf0423b6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:17.769689Z",
     "iopub.status.busy": "2024-10-01T03:04:17.768795Z",
     "iopub.status.idle": "2024-10-01T03:04:17.774673Z",
     "shell.execute_reply": "2024-10-01T03:04:17.773603Z",
     "shell.execute_reply.started": "2024-10-01T03:04:17.769633Z"
    }
   },
   "outputs": [],
   "source": [
    "# !pip install -U langchain\n",
    "# !pip install -U langchain-openai\n",
    "# !pip install -U langgraph\n",
    "# !pip install -U openai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "bb6b1f09-13cb-4924-a813-0ca4e505e061",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T03:13:16.022797Z",
     "iopub.status.busy": "2024-11-02T03:13:16.022238Z",
     "iopub.status.idle": "2024-11-02T03:13:16.030787Z",
     "shell.execute_reply": "2024-11-02T03:13:16.029088Z",
     "shell.execute_reply.started": "2024-11-02T03:13:16.022744Z"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "from rich.pretty import pprint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3cc537ce-ad92-44bf-b4c9-91bc34536b6d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T02:52:38.754695Z",
     "iopub.status.busy": "2024-11-02T02:52:38.754063Z",
     "iopub.status.idle": "2024-11-02T02:52:40.337150Z",
     "shell.execute_reply": "2024-11-02T02:52:40.335664Z",
     "shell.execute_reply.started": "2024-11-02T02:52:38.754639Z"
    }
   },
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, Field\n",
    "from typing import Literal\n",
    "from langchain_core.tools import tool\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langgraph.graph import MessagesState"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5ad05a6f-7952-41dc-891d-8244b8aadc3a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T02:52:40.555421Z",
     "iopub.status.busy": "2024-11-02T02:52:40.555177Z",
     "iopub.status.idle": "2024-11-02T02:52:40.582494Z",
     "shell.execute_reply": "2024-11-02T02:52:40.580646Z",
     "shell.execute_reply.started": "2024-11-02T02:52:40.555400Z"
    }
   },
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "assert load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a7cde72e-2eb5-408b-921c-fcbc1cab0716",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.117962Z",
     "iopub.status.busy": "2024-10-01T03:04:19.117769Z",
     "iopub.status.idle": "2024-10-01T03:04:19.132928Z",
     "shell.execute_reply": "2024-10-01T03:04:19.131343Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.117943Z"
    }
   },
   "outputs": [],
   "source": [
    "class WeatherResponse(BaseModel):\n",
    "    \"\"\"Respond to the user with this\"\"\"\n",
    "    temperature: float = Field(description=\"The temperature in fahrenheit\")\n",
    "    wind_directon: str = Field(description=\"The direction of the wind in abbreviated form\")\n",
    "    wind_speed: float = Field(description=\"The speed of the wind in km/h\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "6953cbe8-026b-4c30-b4fb-824464479aea",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.135761Z",
     "iopub.status.busy": "2024-10-01T03:04:19.135537Z",
     "iopub.status.idle": "2024-10-01T03:04:19.143477Z",
     "shell.execute_reply": "2024-10-01T03:04:19.142347Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.135742Z"
    }
   },
   "outputs": [],
   "source": [
    "# Inherit 'messages' key from MessagesState, which is a list of chat messages \n",
    "class AgentState(MessagesState):\n",
    "    # Final structured response from the agent\n",
    "    final_response: WeatherResponse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7a92931c-767c-4dc7-aaef-842720d8be3e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.144237Z",
     "iopub.status.busy": "2024-10-01T03:04:19.144035Z",
     "iopub.status.idle": "2024-10-01T03:04:19.153235Z",
     "shell.execute_reply": "2024-10-01T03:04:19.151665Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.144209Z"
    }
   },
   "outputs": [],
   "source": [
    "from typing import get_type_hints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5ef324a6-cf97-4447-87af-b8742bc2e5c2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.154380Z",
     "iopub.status.busy": "2024-10-01T03:04:19.154159Z",
     "iopub.status.idle": "2024-10-01T03:04:19.167723Z",
     "shell.execute_reply": "2024-10-01T03:04:19.166167Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.154362Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'messages': list[typing.Union[langchain_core.messages.ai.AIMessage, langchain_core.messages.human.HumanMessage, langchain_core.messages.chat.ChatMessage, langchain_core.messages.system.SystemMessage, langchain_core.messages.function.FunctionMessage, langchain_core.messages.tool.ToolMessage, langchain_core.messages.ai.AIMessageChunk, langchain_core.messages.human.HumanMessageChunk, langchain_core.messages.chat.ChatMessageChunk, langchain_core.messages.system.SystemMessageChunk, langchain_core.messages.function.FunctionMessageChunk, langchain_core.messages.tool.ToolMessageChunk]],\n",
       " 'final_response': __main__.WeatherResponse}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# messages：chat history，append\n",
    "# final_response: \n",
    "get_type_hints(AgentState)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "39c2b335-21d7-4803-8741-21c81c6bec96",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.168633Z",
     "iopub.status.busy": "2024-10-01T03:04:19.168423Z",
     "iopub.status.idle": "2024-10-01T03:04:19.181736Z",
     "shell.execute_reply": "2024-10-01T03:04:19.180152Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.168615Z"
    }
   },
   "outputs": [],
   "source": [
    "@tool\n",
    "def get_weather(city: Literal[\"nyc\", \"sf\"]):\n",
    "    \"\"\"Use this to get weather information.\"\"\"\n",
    "    if city == \"nyc\":\n",
    "        return \"It is cloudy in NYC, with 5 mph winds in the North-East direction and a temperature of 70 degrees\"\n",
    "    elif city == \"sf\":\n",
    "        return \"It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction\"\n",
    "    else:\n",
    "        raise AssertionError(\"Unknown city\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bdc28b75-9e1e-4218-baec-1a17cc3d0abc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.182813Z",
     "iopub.status.busy": "2024-10-01T03:04:19.182593Z",
     "iopub.status.idle": "2024-10-01T03:04:19.190780Z",
     "shell.execute_reply": "2024-10-01T03:04:19.189571Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.182793Z"
    }
   },
   "outputs": [],
   "source": [
    "tools = [get_weather]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "47d948f0-59a1-4890-b2b2-87c52f054914",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.191915Z",
     "iopub.status.busy": "2024-10-01T03:04:19.191659Z",
     "iopub.status.idle": "2024-10-01T03:04:19.262918Z",
     "shell.execute_reply": "2024-10-01T03:04:19.261333Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.191893Z"
    }
   },
   "outputs": [],
   "source": [
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b2a6b2ca-b934-4e9c-9864-354cb7752b6f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.263761Z",
     "iopub.status.busy": "2024-10-01T03:04:19.263563Z",
     "iopub.status.idle": "2024-10-01T03:04:19.357463Z",
     "shell.execute_reply": "2024-10-01T03:04:19.355903Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.263743Z"
    }
   },
   "outputs": [],
   "source": [
    "model_w_output = llm.with_structured_output(WeatherResponse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "38daf7fb-6525-4dc4-88d5-fd68bf21445c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:19.359421Z",
     "iopub.status.busy": "2024-10-01T03:04:19.359170Z",
     "iopub.status.idle": "2024-10-01T03:04:20.402893Z",
     "shell.execute_reply": "2024-10-01T03:04:20.401772Z",
     "shell.execute_reply.started": "2024-10-01T03:04:19.359402Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WeatherResponse(temperature=65.0, wind_directon='NW', wind_speed=15.0)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_w_output.invoke(\"what's the weather in SF?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "cae3253a-3b73-4f06-ab08-00261aed0e04",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:20.403819Z",
     "iopub.status.busy": "2024-10-01T03:04:20.403585Z",
     "iopub.status.idle": "2024-10-01T03:04:20.411606Z",
     "shell.execute_reply": "2024-10-01T03:04:20.410077Z",
     "shell.execute_reply.started": "2024-10-01T03:04:20.403798Z"
    }
   },
   "outputs": [],
   "source": [
    "# update langchain-openai 等\n",
    "model_w_output = llm.with_structured_output(WeatherResponse, strict=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f8c253ef-0a62-4356-a56e-540b2fb1c2f4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:20.413473Z",
     "iopub.status.busy": "2024-10-01T03:04:20.413037Z",
     "iopub.status.idle": "2024-10-01T03:04:27.275791Z",
     "shell.execute_reply": "2024-10-01T03:04:27.273609Z",
     "shell.execute_reply.started": "2024-10-01T03:04:20.413437Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WeatherResponse(temperature=60.0, wind_directon='NW', wind_speed=15.0)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_w_output.invoke(\"what's the weather in SF?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "87d97158-7272-433f-a507-3d9c6870977a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:27.278596Z",
     "iopub.status.busy": "2024-10-01T03:04:27.277923Z",
     "iopub.status.idle": "2024-10-01T03:04:28.105613Z",
     "shell.execute_reply": "2024-10-01T03:04:28.103560Z",
     "shell.execute_reply.started": "2024-10-01T03:04:27.278547Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WeatherResponse(temperature=65.0, wind_directon='NW', wind_speed=10.0)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_w_output.invoke(('Human', \"what's the weather in SF?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7c10b1c0-359c-4754-b77f-b103846c4faf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:28.108305Z",
     "iopub.status.busy": "2024-10-01T03:04:28.107741Z",
     "iopub.status.idle": "2024-10-01T03:04:28.119613Z",
     "shell.execute_reply": "2024-10-01T03:04:28.117474Z",
     "shell.execute_reply.started": "2024-10-01T03:04:28.108257Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"./imgs/structure_output_option1.png\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='./imgs/structure_output_option1.png', width=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "60e9a2bc-2277-4f57-aeaa-1c6d63c66d40",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:28.122251Z",
     "iopub.status.busy": "2024-10-01T03:04:28.121694Z",
     "iopub.status.idle": "2024-10-01T03:04:28.139241Z",
     "shell.execute_reply": "2024-10-01T03:04:28.137275Z",
     "shell.execute_reply.started": "2024-10-01T03:04:28.122189Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"./imgs/structure_output_option2.png\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='./imgs/structure_output_option2.png', width=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0a172370-69b1-4bbe-b5c7-33476ceecba1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:28.141633Z",
     "iopub.status.busy": "2024-10-01T03:04:28.141109Z",
     "iopub.status.idle": "2024-10-01T03:04:28.153449Z",
     "shell.execute_reply": "2024-10-01T03:04:28.151522Z",
     "shell.execute_reply.started": "2024-10-01T03:04:28.141586Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"./imgs/react-output.png\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='./imgs/react-output.png', width=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2be1a43-1909-42a1-9370-b8be5bf145fa",
   "metadata": {},
   "source": [
    "- ReAct agent\n",
    "    - a model node and a tool-calling node\n",
    "- 增加一个 respond 节点，做结构化的输出处理；"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99f464b6-b417-4148-ba52-99f4e07e4075",
   "metadata": {},
   "source": [
    "### basic usage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a620976d-b163-48ab-8d8b-d92a3f630451",
   "metadata": {},
   "source": [
    "- bind_tools：response model as a function/tool 返回的是函数调用，即函数名及参数；\n",
    "- with_structured_output:\n",
    "    - 内部读源码可知，with_structured_output 是对 bind_tools 的进一步封装；\n",
    "    ```\n",
    "    def with_structured_output(\n",
    "        self,\n",
    "        schema: Optional[_DictOrPydanticClass] = None,\n",
    "        *,\n",
    "        method: Literal[\n",
    "            \"function_calling\", \"json_mode\", \"json_schema\"\n",
    "        ] = \"function_calling\",\n",
    "        include_raw: bool = False,\n",
    "        strict: Optional[bool] = None,\n",
    "        **kwargs: Any,\n",
    "    ) -> Runnable[LanguageModelInput, _DictOrPydantic]:\n",
    "    ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "cf6e2062-5933-4487-b7c8-f93d5e71e19a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T03:29:39.309486Z",
     "iopub.status.busy": "2024-11-02T03:29:39.308904Z",
     "iopub.status.idle": "2024-11-02T03:29:39.317447Z",
     "shell.execute_reply": "2024-11-02T03:29:39.315636Z",
     "shell.execute_reply.started": "2024-11-02T03:29:39.309435Z"
    }
   },
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from pydantic import BaseModel, Field\n",
    "from typing import Literal, Optional\n",
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.messages import HumanMessage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "e62543b4-33f1-4af9-bf63-78616cac2edb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T03:30:20.821039Z",
     "iopub.status.busy": "2024-11-02T03:30:20.820466Z",
     "iopub.status.idle": "2024-11-02T03:30:22.115466Z",
     "shell.execute_reply": "2024-11-02T03:30:22.113863Z",
     "shell.execute_reply.started": "2024-11-02T03:30:20.820988Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Joke</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">setup</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Why was the cat sitting on the computer?'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">punchline</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Because it wanted to keep an eye on the mouse!'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">rating</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7</span>\n",
       "<span style=\"font-weight: bold\">)</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;35mJoke\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33msetup\u001b[0m=\u001b[32m'Why was the cat sitting on the computer?'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mpunchline\u001b[0m=\u001b[32m'Because it wanted to keep an eye on the mouse!'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mrating\u001b[0m=\u001b[1;36m7\u001b[0m\n",
       "\u001b[1m)\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define the schema\n",
    "class Joke(BaseModel):\n",
    "    setup: str = Field(description=\"The setup of the joke\")\n",
    "    punchline: str = Field(description=\"The punchline of the joke\")\n",
    "    rating: Optional[int] = Field(description=\"How funny the joke is, from 1 to 10\")\n",
    "\n",
    "# Initialize the language model\n",
    "llm = ChatOpenAI(model=\"gpt-4o-2024-08-06\")\n",
    "\n",
    "# Apply with_structured_output\n",
    "structured_llm = llm.with_structured_output(Joke)\n",
    "\n",
    "# Generate structured output\n",
    "response = structured_llm.invoke(\"Tell me a joke about cats\")\n",
    "pprint(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "5aded2e8-31aa-45b2-9006-556871aae968",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-02T03:32:58.585451Z",
     "iopub.status.busy": "2024-11-02T03:32:58.584994Z",
     "iopub.status.idle": "2024-11-02T03:33:03.971270Z",
     "shell.execute_reply": "2024-11-02T03:33:03.969647Z",
     "shell.execute_reply.started": "2024-11-02T03:32:58.585411Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Response</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">quote</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Life is a purpose, not a means.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">quote_cn</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'生命是目的，不是手段。'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Albert Einstein'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author_cn</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'阿尔伯特·爱因斯坦'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author_birthdate</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">datetime</span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">.datetime</span><span style=\"font-weight: bold\">(</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1879</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">14</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #808000; text-decoration-color: #808000\">tzinfo</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">TzInfo</span><span style=\"font-weight: bold\">(</span>UTC<span style=\"font-weight: bold\">))</span>\n",
       "<span style=\"font-weight: bold\">)</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;35mResponse\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mquote\u001b[0m=\u001b[32m'Life is a purpose, not a means.'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mquote_cn\u001b[0m=\u001b[32m'生命是目的，不是手段。'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor\u001b[0m=\u001b[32m'Albert Einstein'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor_cn\u001b[0m=\u001b[32m'阿尔伯特·爱因斯坦'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor_birthdate\u001b[0m=\u001b[1;35mdatetime\u001b[0m\u001b[1;35m.datetime\u001b[0m\u001b[1m(\u001b[0m\u001b[1;36m1879\u001b[0m, \u001b[1;36m3\u001b[0m, \u001b[1;36m14\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[33mtzinfo\u001b[0m=\u001b[1;35mTzInfo\u001b[0m\u001b[1m(\u001b[0mUTC\u001b[1m)\u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m)\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "“生命是目的不是手段”这种思想与德国哲学家伊曼努尔·康德（Immanuel Kant）的伦理学观点有关。康德在其道德哲学中提出，人应该被视为目的，而不是仅仅作为实现其他目的的手段。这一观点是康德“目的王国”概念的一部分，强调了人类尊严和道德行为的基础。康德的这种思想主要体现在他的著作《道德形而上学基础》中。\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Response</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">quote</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Act only according to that maxim whereby you can, at the same time, will that it should become a universal law.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">quote_cn</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'只根据你同时能愿意它成为普遍法则的准则行事。'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Immanuel Kant'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author_cn</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'伊曼努尔·康德'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">author_birthdate</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">datetime</span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">.datetime</span><span style=\"font-weight: bold\">(</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1724</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">22</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #808000; text-decoration-color: #808000\">tzinfo</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">TzInfo</span><span style=\"font-weight: bold\">(</span>UTC<span style=\"font-weight: bold\">))</span>\n",
       "<span style=\"font-weight: bold\">)</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1;35mResponse\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mquote\u001b[0m=\u001b[32m'Act only according to that maxim whereby you can, at the same time, will that it should become a universal law.'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mquote_cn\u001b[0m=\u001b[32m'只根据你同时能愿意它成为普遍法则的准则行事。'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor\u001b[0m=\u001b[32m'Immanuel Kant'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor_cn\u001b[0m=\u001b[32m'伊曼努尔·康德'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mauthor_birthdate\u001b[0m=\u001b[1;35mdatetime\u001b[0m\u001b[1;35m.datetime\u001b[0m\u001b[1m(\u001b[0m\u001b[1;36m1724\u001b[0m, \u001b[1;36m4\u001b[0m, \u001b[1;36m22\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[33mtzinfo\u001b[0m=\u001b[1;35mTzInfo\u001b[0m\u001b[1m(\u001b[0mUTC\u001b[1m)\u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m)\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "class Response(BaseModel):\n",
    "    \"\"\"Structured response model for a famous quote\"\"\"\n",
    "    quote: str = Field(description=\"A famous quote by an individual\")\n",
    "    quote_cn: str = Field(description=\"The Chinese version of the famous quote\")\n",
    "    author: str = Field(description=\"The name of the author of the quote\")\n",
    "    author_cn: str = Field(description=\"The Chinese name of the author\")\n",
    "    author_birthdate: datetime = Field(description=\"The birthdate of the author in datetime format\")\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-2024-08-06\", temperature=0)\n",
    "model_w_output = llm.with_structured_output(Response)\n",
    "resp = model_w_output.invoke(\"类似`生命是目的不是手段`这种含义的原文是什么，是谁说的\")\n",
    "pprint(resp)\n",
    "\n",
    "llm = ChatOpenAI(model=\"gpt-4o-2024-08-06\", temperature=0)\n",
    "ai_message = llm.invoke(\"类似`生命是目的不是手段`这种含义的原文是什么，是谁说的\")\n",
    "print(ai_message.content)\n",
    "resp = model_w_output.invoke([HumanMessage(ai_message.content)])\n",
    "pprint(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e519ca33-c7e2-400f-8ffb-c5e4353050c0",
   "metadata": {},
   "source": [
    "### Option 1: Bind output as tool¶"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "5026f3d6-df63-4d47-8199-1b986c4d9573",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:28.156015Z",
     "iopub.status.busy": "2024-10-01T03:04:28.155494Z",
     "iopub.status.idle": "2024-10-01T03:04:28.168049Z",
     "shell.execute_reply": "2024-10-01T03:04:28.166084Z",
     "shell.execute_reply.started": "2024-10-01T03:04:28.155969Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"./imgs/structure_output_option1.png\" width=\"500\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url='./imgs/structure_output_option1.png', width=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ffdcfdf-2a20-4975-8496-448f5791e1c6",
   "metadata": {},
   "source": [
    "- only one `LLM`\n",
    "- setting `tool_choice` to `any` when we use `bind_tools` which forces the LLM to select at least one tool at every turn, but this is far from a fool proof strategy. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4cfd02eb-1ffb-4fca-af27-9b31142befbb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:28.178611Z",
     "iopub.status.busy": "2024-10-01T03:04:28.178034Z",
     "iopub.status.idle": "2024-10-01T03:04:28.205083Z",
     "shell.execute_reply": "2024-10-01T03:04:28.203091Z",
     "shell.execute_reply.started": "2024-10-01T03:04:28.178563Z"
    }
   },
   "outputs": [],
   "source": [
    "from langgraph.graph import StateGraph, END\n",
    "from langgraph.prebuilt import ToolNode\n",
    "\n",
    "# pydantic model as a tool\n",
    "tools = [get_weather, WeatherResponse]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "9e8612bf-68fd-4e9e-bd20-72141e151aa0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:35.669706Z",
     "iopub.status.busy": "2024-10-01T03:04:35.669260Z",
     "iopub.status.idle": "2024-10-01T03:04:35.683022Z",
     "shell.execute_reply": "2024-10-01T03:04:35.681202Z",
     "shell.execute_reply.started": "2024-10-01T03:04:35.669666Z"
    }
   },
   "outputs": [],
   "source": [
    "# Force the model to use tools by passing tool_choice=\"any\"    \n",
    "model_with_resp_tool = llm.bind_tools(tools, tool_choice=\"any\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "55fb6b22-8f4b-4e6a-a158-d245bb8e718f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:38.441553Z",
     "iopub.status.busy": "2024-10-01T03:04:38.440962Z",
     "iopub.status.idle": "2024-10-01T03:04:38.450549Z",
     "shell.execute_reply": "2024-10-01T03:04:38.448569Z",
     "shell.execute_reply.started": "2024-10-01T03:04:38.441501Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that calls the model\n",
    "def call_model(state: AgentState):\n",
    "    response = model_with_resp_tool.invoke(state['messages'])\n",
    "    # We return a list, because this will get added to the existing list\n",
    "    return {\"messages\": [response]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "28c9117a-e768-4fac-a22b-6f23bf5b0478",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:41.366806Z",
     "iopub.status.busy": "2024-10-01T03:04:41.366195Z",
     "iopub.status.idle": "2024-10-01T03:04:41.375343Z",
     "shell.execute_reply": "2024-10-01T03:04:41.373713Z",
     "shell.execute_reply.started": "2024-10-01T03:04:41.366756Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that responds to the user\n",
    "def respond(state: AgentState):\n",
    "    # Construct the final answer from the arguments of the last tool call\n",
    "    response = WeatherResponse(**state['messages'][-1].tool_calls[0]['args'])\n",
    "    # We return the final answer\n",
    "    return {\"final_response\": response}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "18aec5a0-6a13-4f58-b201-b4bb8c89b00b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:43.769580Z",
     "iopub.status.busy": "2024-10-01T03:04:43.769000Z",
     "iopub.status.idle": "2024-10-01T03:04:43.780309Z",
     "shell.execute_reply": "2024-10-01T03:04:43.778404Z",
     "shell.execute_reply.started": "2024-10-01T03:04:43.769531Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that determines whether to continue or not\n",
    "def should_continue(state: AgentState):\n",
    "    messages = state[\"messages\"]\n",
    "    last_message = messages[-1]\n",
    "    # If there is only one tool call and it is the response tool call we respond to the user\n",
    "    if len(last_message.tool_calls) == 1 and last_message.tool_calls[0]['name'] == \"WeatherResponse\":\n",
    "        return \"respond\"\n",
    "    # Otherwise we will use the tool node again\n",
    "    else:\n",
    "        return \"continue\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "63cf07df-5427-41e8-93fb-27daed86248b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:45.341494Z",
     "iopub.status.busy": "2024-10-01T03:04:45.340916Z",
     "iopub.status.idle": "2024-10-01T03:04:45.372197Z",
     "shell.execute_reply": "2024-10-01T03:04:45.370211Z",
     "shell.execute_reply.started": "2024-10-01T03:04:45.341442Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<langgraph.graph.state.StateGraph at 0x7fcae05147d0>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define a new graph\n",
    "workflow = StateGraph(AgentState)\n",
    "\n",
    "# Define the two nodes we will cycle between\n",
    "workflow.add_node(\"agent\", call_model)\n",
    "workflow.add_node(\"respond\", respond)\n",
    "workflow.add_node(\"tools\", ToolNode(tools))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "716591e1-93c2-4f6a-a538-89dabf0a1f0a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:46.835920Z",
     "iopub.status.busy": "2024-10-01T03:04:46.835358Z",
     "iopub.status.idle": "2024-10-01T03:04:46.847613Z",
     "shell.execute_reply": "2024-10-01T03:04:46.845501Z",
     "shell.execute_reply.started": "2024-10-01T03:04:46.835872Z"
    }
   },
   "outputs": [],
   "source": [
    "# Set the entrypoint as `agent`\n",
    "# This means that this node is the first one called\n",
    "workflow.set_entry_point(\"agent\")\n",
    "\n",
    "# We now add a conditional edge\n",
    "workflow.add_conditional_edges(\n",
    "    \"agent\",\n",
    "    should_continue,\n",
    "    {\n",
    "        \"continue\": \"tools\",\n",
    "        \"respond\": \"respond\",\n",
    "    },\n",
    ")\n",
    "\n",
    "workflow.add_edge(\"tools\", \"agent\")\n",
    "workflow.add_edge(\"respond\", END)\n",
    "graph = workflow.compile()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "8ed5c23f-ca9a-48e6-9ee2-75d263f264be",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:49.342302Z",
     "iopub.status.busy": "2024-10-01T03:04:49.341725Z",
     "iopub.status.idle": "2024-10-01T03:04:49.396488Z",
     "shell.execute_reply": "2024-10-01T03:04:49.394606Z",
     "shell.execute_reply.started": "2024-10-01T03:04:49.342252Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Edge(source='__start__', target='agent', data=None, conditional=False),\n",
       " Edge(source='respond', target='__end__', data=None, conditional=False),\n",
       " Edge(source='tools', target='agent', data=None, conditional=False),\n",
       " Edge(source='agent', target='tools', data='continue', conditional=True),\n",
       " Edge(source='agent', target='respond', data=None, conditional=True)]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph.get_graph().edges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "54bd4a19-e309-4343-a9ef-f387de632961",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:50.468000Z",
     "iopub.status.busy": "2024-10-01T03:04:50.467441Z",
     "iopub.status.idle": "2024-10-01T03:04:51.247770Z",
     "shell.execute_reply": "2024-10-01T03:04:51.246274Z",
     "shell.execute_reply.started": "2024-10-01T03:04:50.467952Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAFlANkDASIAAhEBAxEB/8QAHQABAAIDAQEBAQAAAAAAAAAAAAYHBAUIAwIBCf/EAFQQAAEEAQIDAgcKCggCCAcAAAEAAgMEBQYRBxIhEzEUFSJBUVaUCBYXNlRhcZHR0yMyU1V0gZWytNI1QlJ1k6GxsyY3CRgkV2JzgsE0Q2NykqLU/8QAGwEBAQADAQEBAAAAAAAAAAAAAAECAwQFBgf/xAA4EQEAAQEEBgYJAwUBAAAAAAAAAQIDESFRBBITMZHRM2FxobHBBRQjQUJSYoGSFdLwMkNTouHx/9oADAMBAAIRAxEAPwD+qaIiAiIgIiICIiAiIgIiIC+JZWQRl8j2xsb3ucdgP1rU5rL2WWo8Xi2MkykzO07SZpdDVj327SQAgnruGsBBeQeoAc5uJFoDFTSixlY3Z+51Pb5PaUN36eRHtyMG3TyWj599yVuiimIvrm7xW7NsXaowzCQ7LUQR5jZZ9q/PfVhfzxQ9pZ9q/BpTCNAAw9AAdABVZ9i/ferhfzPQ9mZ9iy9j19y4Hvqwv54oe0s+1PfVhfzxQ9pZ9qe9XC/meh7Mz7E96uF/M9D2Zn2J7Hr7jA99WF/PFD2ln2p76sL+eKHtLPtT3q4X8z0PZmfYnvVwv5noezM+xPY9fcYHvqwv54oe0s+1ZFTM4++/kq3q1l39mGZrj/kVj+9XC/meh7Mz7Fj29D6dvx8ljBY2UebmqM3HXfodtx167hPY9fcmDeIos7GXdHtNjGy28limDeXGTPM8sbf7UD3HmO35NxO46M5duV0jqW4b9WGzXkbNBMwPjkYdw5pG4IWFdGrjTN8Fz2REWpBERAREQEREBERAREQEREBERBF9AbX8bazb9nT5azJPzeiFriyFvzARtadh05nOPeSTKFGOGw7DR9Ok7cS490lGQEbEOikczf6CGgj0gg+dSddGkdLVHX3e7uWd4tNrDWOG0Bpy7n9QZCPGYim0OnsygkN3cGtAABLiXOAAAJJIAG5W5UD45YjEZzhhmKecwuYz2OeYS+pp+Nz77XCZhZLCGkO5o3BsnTr5B6O7jzoiOuvdUaY0vpnTecxsV7L0svnocM8jG3GSV9yO1cYuwMnO1pBbGWgvJ8nfYhSPVXuhtBaIr4ybOZezj2ZGo2/CJMVcLmQHukmaIiYR379qG7bHfbZUjbPEPUHCjF5TMYjUOfh0xr2lkqHhuNEOavYiB7CZJKrQC6UF8g25WueGb8oJ67LitmNRa81PLHZxHEKPSGQ0+PEuNwFWek6fIOfKyVt9zS10OzRDytmc2Mtc4nc7hBc+quOeh9GW8VVymbAs5am6/j4adWe265A0s3dEIWP5z5bTyt3JG7gNgSI9pD3RmG1bxg1DoRlHI15ce2r4LafjbgbYdJFJLJ2hMIbAGhgDS9w5yTyk9yrnghpPNQar4H2clgMnTbhuHtnGW5LtKSMVLbJKkfZuLhs1xEcnL/aaCRuOqmeFsZDQ/umNbvvaezVnG6tr4nxflaFF9ipG6BkscrZ5GjaEguafK2BBQXgiIgKL6V2xmd1BhW7CCCWO9XYN/IjnDiW/4rJiPQHAeZShRjCDwvXOprrd+zihqY7fbYF8YkldsfP0stH0ghdFn/RXHV5x/wBWN0pOiIudBERAREQEREBERAREQEREBERBG78MumcrZy9eF8+Pt8rshBC1z5GPaA0TsaPxvJAa5oG5DGkdQQ781DpDSXFLEVG5vEYnVOMY/tq/hkEdqIO2LeZu4I37xuPnUlUfvaGxVu5LchbPjLspLpLGNsPrulcRtzPDCGvO23VwJ6D0Bb4qori6vCc/5/Opd+9Fv+rXwn/7t9LfsiD+VbrSfB/Q2g8m7I6b0hhMFfdEYXWsdQigkLCQS3ma0HYloO3zBe50TP5tUZ5o9HbRH/WJPeTY9as9/jQ/dK7Oz+fukujNKEUX95Nj1qz3+ND90onxDx+V0xXwD6WqcwXXs1ToS9tLCfwUsnK/b8GPK27v9E2dn8/dJdGa1Fi5TF083jbeOyFWG7QtxOgsVrDA+OWNwIcxzT0IIJBB9K0PvJsetWe/xofuk95Nj1qz3+ND90mzs/n7pLozaCP3N/CmGRr2cONLse0hzXNxMAII7iDyr7qe524W0LcNqtw80zXswvbJFLHioWuY4HcOBDehBG+63nvJsetWe/xoful+nQjJhy287nLce2xYbxhDh85iDD/mmpZx8fdJdGbLzOoxBZOLxnZXc49u7a5d5MDT3STEfiM9Hndts3frtl4HDR4HGMqse6Z/M6WWZ/40sj3Fz3n6XEnbzd3mX3iMJQwNTwbHVIqcHMXFsTduZx73OPeSfOT1KzljVXF2pRu8TsERFpQREQEREBERAREQEREBERAREQEREBERAVe8ZSBU0huSP+KMb3f+d9KsJV7xl38E0htt8Z8b3gflfnQWEiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKvOMw3p6P6hv/FGN7x/9ZWGq84z7eB6P39aMZ5t//nILDREQEREBERAREQEREBERAREQEREBERAREQEREBEUd1BqexSujHYqnHfyXIJZBPKYoYGEkNL3BrjuSDs0A77HcgbE7KKKrSdWlbr0iRQjx7rD5Bg/a5vu08e6w+QYP2ub7tdHqteccYLk3XF3uyfde2uDWv8AEaWvaFku1KtulnKeUGSaxtxkbt3M5DC7kIeHN33PcD59l01491h8gwftc33aqP3QXAW17oqHTTNQUsRA7C322myQWpeaaE7drAT2fRr9m9fNsnqteccYLlw8Htd3eJ/DPT+q8hg36bsZev4U3GyT9u6KNzj2RL+Vu/Mzkf3Dbm267bqZKCwZbVlaGOGHGYGKKNoYyNlmYNa0DYAARdAF9+PdYfIMH7XN92nqteccYLk3RQjx7rD5Bg/a5vu19M1Jqmse0s4jG2oW9Xx07jxMR5+QPjDXH0AuaD6QnqtpnHGC5NUWLjMlWzGPr3qkna1rDBJG/YgkH0g9QfSD1B6FZS5JiYm6UERFAREQEREBERAREQEREBERAREQFA6x315qjfzCoP1dmftKnigdX4+ao+ir/tFdui/H2ecMo3S3KIi3MRERARFg5fOY/AwQzZK7BRimnjqxvsSBgfLI4Mjjbv3uc4gADqSUGciIgx+Fx30ZW+azbA+YCzKApWonwu+Jlf8ASrn8VKpYuXSentO2fFZ3yIiLmQREQEREBERAREQEREBERAREQFA6vx81R9FX/aKnigdX4+ao+ir/ALRXbovx9nnDKN0tyudeP93Kal1fewmlbOpW5rC4Txjbkx2oTiaNNr3Sdk94bG8zykxv8gjk5W9SN10UoXq/g1o7Xuchy+dwrb1+ODwUyCxLE2aHmLhFMxjw2Zm5J5JA5vU9OpWyqL4Yue8nxIv6nqcMbOtNT6g01p7M6KZkm3dNPlhkuZgiMva4wtLujHc7I/xXFxBB22WvxF7idnZuHXD+4chDkWaRGdyMMmpJ8RbtWHWDGe0sNimlJY3lLohygGXr0aGqzOJXud5bcWmqmjcJhfFuHqS044shnMrQmhjc8PDWTVpCXRgj8R4O3TlLQNlIcH7n3GZXh5pjC8QJX6tzmFEro8y21YgsxF73HkjnbIJuUNLY+r93BgLuq16s3iuvE/ECDUXCvSGr9UZCochezLJ34bLvdNYpxwCWCOawI4i+Ru3KZAxrthuCC4lRXWFS1qLh/YwmXzmauwad4rU8PUuPyUzbPgrp6zmiSVrg57mCdwa9xLmlrSDuAV0/jOFelsO/TLqeKbA7TYnGL5ZpP+z9s0tlPV3llwJ3L+Y7nfv6rxv8INIZTBaiw1vDR2MbqC6clkoJJpD21kiMdqDzbscOyjI5C3YtBGx6q6siSYTExYHE1MfDPasxVoxG2W7ZfYmeB53yPJc8/O4krNWr0zpuhpDB1cRi45YqNYOEbZ7Elh43cXHeSRznu6k95K2i2DG4XfEyv+lXP4qVSxRPhd8TK/6Vc/ipVLFzaT09p2z4rO+RERcyCIiAiIgIiICIiAiIgIiICIiAoHV+PmqPoq/7RUvzOaoaex0t/JW4qVOMta6aZ3KOZzg1jR6XOc5rQB1JcAASQFEbdPNjN5TMVMK4skkFV1OWyxsthkY8mxGdy1vMXObyOIJa1jt2ndp69GqiJqpmd8Xd8T5LDcItJ42z3qZlfaqX36eNs96mZX2ql9+uzU+qPyjmtzdotJ42z3qZlfaqX36w8nqzLYhtZ1nRuZAsTsrR9nJVk8t52bvyzHlG/e47AechNT6o/KOZck6LSeNs96mZX2ql9+njbPepmV9qpffpqfVH5RzLm7RRnMary2CxN3JXNH5dlSnA+xM5k1SRwYxpc4hrZiXHYHoASfMFWnDr3XGieL+QZi9K367MnM9sUUeYsR1d3O6DlY53PId/6rRue7cJqfVH5RzS5dHC74mV/wBKufxUqli1uncKzT2Fq4+OR0whaeaV/e9xJc5x+lxJ/WtkvOtqortaq43TM+JO8REWlBERAREQEREBERAREQERY+QyFXE0LN69Zhp0q0bpp7NiQMjijaCXPc49GtABJJ6ABBkLVX86IL1ejUqzX7Mz3xudEPwNYtj5953/ANQHmYAOrjzghpaHEYRnyOpy9tV0+IxX/Zp4MjG5hmtsPlyMEb2nsmkcjC53l+VJsGFrXnbYvD0cJWfXx9OCjC+WSd8deMMDpJHF8jzt3uc5znOcepJJO5KDCw+CmhmbkMnaNzLSVooZ+yc9lVrm7uJhhc5wZu4k7klxAaC48rdtyiICIiAo5rWZ8MOHLJcnFzZWq0+LIucuBf1bL6Ij/XPmCkajutp/BqmLf2+SgBylRm+MZzudzStbyyDzRHfyz5m7lBIkREHy9jZWOY9oexw2LXDcEegri7hR7gZnB7iXluIMsWO1bNRy77OCwDido6m5IkEj+VguNJBjDwYgY+r2ukbNB2miDW6e1Fj9VYiDJ4ux4RTm5gC5jo3se1xa+N7HAOjkY5rmOY8BzHNc1wBBA2Sh+ocBkMPl5dS6bY6a49oGRw3OGx5RjW7NcwuIbHZaAA2QkNe0COQ7COSHe6c1HQ1XiYsjjZjLXeXMLXsMckT2ktfHIxwDmPa4FrmOALSCCAQg2aIiAiIgIiICIiAiIgIi1uYyr6PZVqsQsZK02TwWJ/MIy5rC7eR7WuMbNw1pdsdi5o2JIBD1yuYrYeOu6z2pNixHWiZDC+V7nvdsPJYCdh1c53c1rXOcQ1pIwMfirl6zWyOYcI7cInZHSqzuNdrHu8kvB255AxrRuRsC54b0JJyMRg24+xYvTyeEZS5HCy1YbztY4xt2AjY5zuzZuXuDQT1e4kkklbRAREQFjy368DyySZrHjvBKyFFM7/Skv0N/0CCQeNanyhn1p41qfKGfWqVw3HDROodUe9/GZrw7JGWSBphqzurvkYCXsbY5Oyc5oa7cB5PQr503x00Lq/UUeExGfjt5CYyNrjsJWRWjHv2ghmcwRzcuxJ7NztgCfMguzxrU+UM+taPWN+GXEQmHIXKz479OQuxrQ6VzW2Yy5hH5NzQWv9DC4juVU4vj/oHNZanjqeeEti3adQheak7YXWWuc0wGV0YY2Tdp2YXBx6EAgjfUac90Jh9Qax1xp+WpkajtOTvibPHjbconZHA2WV+4h5WkEua1m5LwAW8wc1B0T41qfKGfWnjWp8oZ9a53017oDTzNF6Zyeos5SfezzbclIYihcc222GbkcIonRmUvaHR7sLeYnmLQWgkSnF8W9JZqpp+zSzDLEOetSUceWwyAyTxte58bgW7xuaIn7h4bsW7d+wQW/wCNanyhn1p41qfKGfWqlyPFLSuIOpPDsxFUGnHQx5R0zHtEDpWNfE0EjZ5c17dgzmO7gO/ov3RHFDTPEU3GYHJG1PT5fCas9eWtPCHAlpdFK1rwDsdjtsdjseiC4YpWTsD43B7D3EKJ6lwd/DZGfU2nIDYyBYPD8SHBrcpGwdOUuIayw0dGPJDXDZjyByPi3+C/ouH/ANX+pWwQa7T2foapw1TK4yfwilZZzMcWOY4ddnNexwDmPaQWuY4BzXAggEELYqvbB94HEuk6P8HgtWyOgkia0BlfJsjdI2QegTxRva7fpzwx7eVId7CQEREBERAREQEREGBmso3E0HygwusPPZ1oZ52wieY/iRhx7i49O4n5ivPCYo0GT2J9zkLrmzWj275WNfyNbyx8/wCLGOXo1oaNy5xHM5xOtsujy2va1N8uMsR4imL7qkkXaW4J5nPihna49I2ljLbNx1dzOG4AIMlQEREBERAUI1rTlyDMnVgmNaaeu6Jkw743OZsHfqJ3U3Udy+Ns2L8kkcRcw7bEEegIOYOFORydLhRjeFNvR+osDn6+Klw82Qbj3eLo5BC9vhIsg8jmvds7yd3cz+o7ytJp2nnNT4Tg1oqLRmawOR0ffpWcvevUjDTgZVgfG8Qz/izGZzthyF3RxLtl1d4mufkHfWE8TXPyDvrCDlOno/Ox+5503QOEyLcpX1wy66r4JIJo4hnHydqW7bhvZnn5ttuU777dVO9DT3tI8cOIGOv4LLmtqPIVshQytek+SkY20o43iSZvkxuD4XDZ2xO7du9Xj4mufkHfWE8TXPyDvrCDlLg7o/O4y5wNdcwmRqjG1NRtuusVJGeCulmYYhJuPILwCW77cw7t14xaazunr8WoZNPZaxTxHE3KZKatVpSSWHU54JY22IogOaRnNID5AO4JI32K6tdi7cVhjDXmJl32IG7WkDznzb/OvbxNc/IO+sIOMtY6S1Lr7K671HU0vqOvRi1ZhMy3H8slC9fpwUhFIa7g5pDwTzgBwcCwNPK7oLk4MYDT0+dymo8bhdaUch4NHQdc1lPcdLNFzF/JGyzI5wa13UnYDd3Tfcq6fE1z8g76wnia5+Qd9YQb7Bf0XD/6v9StgsPEQvr0I45G8rxvuD9JWYgrzjztDw7feBDZ8dk8behcd9w+O7A7Ybf2gC35w4g9CVYarzjdte0/gsGCe1zOocbWa0DfmZHZZZmH+DXm+jvVhoCIiAiIgIiICIvC7dr4ylYuXLEVSpXjdLNPO8MjjY0buc5x6AAAkk9AAg0uNn59cZ2Lw2jLyVKZ8Eij2sw7un8qV3na7byB5uV/pUhVZYHi7ofIa+yUFbXWkLcluKlXq1ql6A25JS6UcpcHfhNy5oYwbkHm6eUFZqAiIgIiICIiAiIgIiINbn8HX1BjxXsR85jljswntHxlksbw+N3Mwh2wc0bgHqNwdwSD+4DJvyuMilsMhgvMAjt1oJxM2CYAc8fMO/YnzgHYgkDuWxUYyctbSOcblJJaOPxmTljr3D4KRLNde6KGu90renUAReWOv4IBw25XBJ0WpwOq8PqiXLR4jIQZB2Kuvx1013czYbLGsc+InuLmh7QQN9ju07EEDbICIsXK5SphMZcyN+dlWjThfYsTyHZscbGlznE+gAE/qQQe2/3z8bKNZj3GnpbGuuTNH4pt2yYoevpZDFZ3HosMPTpvYShHCTGWo9NTZzJ15KuY1HYdl7cEzgXwdo1rYYDt03ihZDGdunMxx86m6AiIgLS5jW2ntP2hWyecx2Pskc3Y2bTGP29PKTvss3NXHY/D3rTAC+CCSVoPpa0kf6KI6SqR1sBSkA5p7MTJ55ndXzSOaC57iepJJ/V3dwXXY2VNVM117upYzlsvhS0d604j22P7U+FLR3rTiPbY/tXsi3bKxynjHJcHj8KWjvWnEe2x/avK5xH0PkKk9WzqPCz1p2OilikuRlr2OGxaRv1BBIWWibKxynjHIwfzv4Ee5q09or3YGUyGRzWOGiNNT+M8Pckts7O09x5q7Gu3G7oj1dt3GMb/AIwX9CvhS0d604j22P7V7Imyscp4xyMHj8KWjvWnEe2x/anwpaO9acR7bH9q9kTZWOU8Y5GDx+FLR3rTiPbY/tW3w2pMTqJsjsXk6mREe3P4LO2Tk3G432PTcdeq1y0OqOXHvxmVhAjuwX6sDZWjynRzTxxSRn0tIfvsdxu1rtt2ja7CzrnVpviZ67/KDCcFgoiLzWIiIgIiINTmdW4TTsjI8pl6OOke3nay1YZG4t323AJ3236brnr3XOPn4z8NzitD8Wcfpq3EJXWMezIMhiyrHN5exllaedreUvHL1Y7n8sdAW2toLlt6Zo5V4D7uUiZdszkeXI97Q7qfQBs0DuDWgDYABSJenNhZUTNNV8zHXEeUssIwcjf9HRfh4XcONV4DVtypgrsOac5jLdljWyjsmAujdvyvbu38ZpIXWfwpaO9acR7bH9q9kWOyscp4xyMHj8KWjvWnEe2x/aoLxN4g6a1ZZwGlYs/jX4rI2TZzFnwlhibTg5XuhLt9uaaQwx8p6mMzEfiFWAibKxynjHIwePwpaO9acR7bH9qycfxB0xlbUdanqHGWbEjgxkUVuNznuPcAN+p+ZfC8L1Gvkqsla1CyxXkaWvjkbu1wTZWOU8Y5JglSKN8PL82R0jTksSvnliknrGWQ7ueIpnxgkkkk7MG5PU95UkXFaUTZ1zRPum4nBq9VfFjMfoc37hUe018XMV+iRfuBSHVXxYzH6HN+4VHtNfFzFfokX7gXbY9DPb5HubJEVDad90/au8K7HEjO6Sbp/SEdd5imdlWy2bFgTiBkbYzG1oY95IEj3t223LQ3ylZmIRfKLnTG+7IxYZm2ZPGY6W3Qw1nNQRaZ1BXzLZo4Nu0ie6MNMUmzmnygWkBxDjylSK5x61DiuHmN1Lc0TUfNlMrQx2Pq0c/HYitNtPaxsjZxEANi7YgjYkfjbdVjrQLpRUpk/dJO0X75qWtdMS4XPYilBkK1HHXW3mZKKabweLsZOSM8xmLWFrmjbmB3I6r7scf8vptmpamrdFjA5zGadtakpVIMq21Beggb+Ej7YRt7ORriwEcrhs/cFwCutAuhFUWm+O96zqLBUtTaWGmMfqDGT5XF3jkmWSY4WMkkZOxrAInCOQO6OeNgeu4UB1Bxq1bre5wsylHTd3TOjs1quoKmU8ahs+QquinLWzVmtBbHIAHgFzhs0bgdFNaB02tBrf8Aoep/emO/jYFv1oNb/wBD1P70x38bAuix6SnthlTvhYSIi8diIiICIiCueG3/AC90z/dtf/bapGo5w2/5e6Z/u2v/ALbVI17Nv0tfbPis75EXPnCrjVqmvpziTqLX9GpX0/p3K5UG5Vv9vNEK8gAqsiEEYc1rQQ2Qu3cdt2jfcb3G+6Dv4rI4+LXmkH6MpZXHWslj7YyLbnMyvF20sczWsaYpBFu/YcwPK4c24XNrQi5kVN6U4+5XKZTSR1BombTeB1cSzC5J+RZYkc8xOmiZYhDR2LpI2uIAc/qNjsVCuHnHPLaU4H8KzNUl1VqTVE9inDYy+V8GiL2Pmf8AhbMjXnmLWBrG7EuPQdya0DplFhYS3cvYelZyFHxXflhY+el2zZuweRu5nO3o7Y9Nx0OyzVmMXhb8Tov029/FzKWqJcLfidF+m3v4uZS1cuk9Pads+Kzvlq9VfFjMfoc37hUe018XMV+iRfuBSHVXxYzH6HN+4VHtNfFzFfokX7gXRY9DPb5HubJUdR9z5fte5lpcNcjlK9PNVWtlhyNMGaKKxHaNiF4Dg0uaHBoIIG43+lXiisxEorPT2P4neK8ucxU0PVyXgZjx4x4sywvsdd3zFzWERnp5Ddz3+UVRGt+FepuHemZchJHp7FXM5rjT09DA4Z0xxdOdlhjTJ5TWEGV5aXhrB0aNtz1XYaLGabxzzqD3PurOJ9jU+d1hlcPjNSWaFShhY8KJZ61AV7QtsfI6RrHSOfM1nMA1uzRsNydxk53g1rviNNqbL6staeqZebSl/TeIo4mWd9aN9pv4SxNJIwOG5bGOVrTyt36uKv1E1YFQ5rgxez2S4a+FT1DjtP4i9jMmxsjhJL4RUjg3h8jYjdrty7l6bdD3KI4fgxxNix/DfTmUv6Xt6e0Tl6lqC/C+wy7bq14pIow6MsLGyBjxvs4gkb7jz9GImrALQa3/AKHqf3pjv42Bb9aDW/8AQ9T+9Md/GwLosekp7YZU74WEiIvHYiIiAiIgrnht/wAvdM/3bX/22qRqOcNv+Xumf7tr/wC21SNezb9LX2z4rO+VEs4GaitY7iVo6/axMmiNXWshfiuxPlGRqzWtncpiLezc1j9yDz7kbdAvGfgnrPiZk8OOJN7BtxeGxl6jDHgHTOkuy2qxrSTy9q0CPaJz9mN5vKfvzdAFfiLm1YRRGnOEGvchkOH9HWOSwEmntESts1ZMV23hWSnigdBA+Zr2hsPK17nENc/d3oCxcVwb1vp/g9X0CaGiNV42rbsRNhznhDY7FJxL4nO5WO5Jmve7fYOGwBDgT06BRNWBEeEmjr/D7hrp3TmUyjs1kMbUZXmuu5vwhHmHMSeUDZo367NG6lyIstwxeFvxOi/Tb38XMpaolwt+J0X6be/i5lLVzaT09p2z4rO+WHmabsjiL1RhAfPBJECfMXNI/wDdRDSVyOxgacIPJZrQsgsQO6Phka0BzHA9QQfrGxHQhTtaXMaK0/qGwLGUweNyM4HKJbVSOR4Ho3cCdlbG1pppmivcdTGRePwV6M9U8J+z4v5U+CvRnqnhP2fF/Kt+1sc54RzMHsi8fgr0Z6p4T9nxfyp8FejPVPCfs+L+VNrY5zwjmYPZFANPcO9LzcYNaU5NPYqSnBjMU+Gq6nEY4nOdc53Nbt0LuVu52G/KO/bpOfgr0Z6p4T9nxfyptbHOeEczB7IvH4K9GeqeE/Z8X8qfBXoz1Twn7Pi/lTa2Oc8I5mD2Wh1OWZGXGYiFwlvT36tgQtPlNihnjlkkcPM0Bm252G7mt33cN9x8FejPVPCfs+L+VbjD6dxWno5I8XjamNZJtztqQNiDthsN+UDfYK7ezonWpvmY6rvOVwjFsURF5rEREQEREFdaDLKem6WIeQy9iomUrMDj5cb2NDdyOnQjZwPcQ4EbghSJZ2Z0nhNRPa/K4ejkntbyNdbrMlIbvvsC4Hpv12Wr+CvRnqnhP2fF/KvTm3sq5mqq+JnqifOGWE4vZF4/BXoz1Twn7Pi/lT4K9GeqeE/Z8X8qx2tjnPCOaYPZF4/BXoz1Twn7Pi/lT4K9GeqeE/Z8X8qbWxznhHMwey8L1+tjKslm3MyvBGN3SSHYBfvwV6M9U8J+z4v5VlY7h/pjEWmWaOncVTsMIcyWClGx7SO4ggbgptbHOeEczB58PKE2N0lTjsRPglkkmsmKQbOYJZnyAEeYgPG4PUKRoi4rSubSua5983k4iIi1oIiICIiCvtMk/Dbrsc27RisRs3r08q7+r6vR9CsFV9plpHG/Xh5NgcThxz9evl3eno6f+/0KwUBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERBXumWEccNeO5XAHE4ccx7j5d3oPr/zCsJV9pkD4bteHZu/inD9Rvv8AjXf1fV+vzKwUBERAREQEREBERAREQEREBERAREQEREBERARFiZbLVMHjrF+/O2tUgbzySP8AMPo7ySdgAOpJAHUqxE1TdG8ZaKkc7xozeTlc3DVoMRT/AKs1tnbWHj08oIaz6Dz/AKlH3a81i7b/AIqtNP8A4adTb/OEr3rP0LpNcX1TFPbM+USuGbo5eF9tl1GwKT4o7hjcIHzsL42v28kuaCCW77bgEEjzhc7+/rWXrZc9kqfcp7+tZetlz2Sp9ytn6FpHz09/7TDNQ3A73TPGfWXus8hpO/pzTNS/NLDR1AY6lkCrVpvlL3RntyGvcJngF3MCSzp37/0IXJGJxlrB66zOsqOVmr6mzEMde9kG1axfNHHtyjYxcre4b8oG+w332ClHv61l62XPZKn3KfoWkfPT3/tMM3R6LnD39ay9bLnslT7lfTNfayjcHe+ixLt/Vlp1S0//AIxA/wCafoWkfPT38jDN0aiqDTPG2zBK2DU1aAQHp4yotc1rP/MiJJA9LmuPztAG6t1j2yMa9jg5rhuHA7gj0rx9J0S10WrVtY38JR9IiLkBERAREQEREBERAREQEREBERAVDcWNSSZ/V0uNa7fH4gtYGb9H2S0Oc8+nla4NHoJer5XMGU5hqXUQk37QZW3vzHzdq4t//UtX0XoSyprt6q5+GMPuu6HgiIvtmsRRPi1kMpiuGWqLmE5/GsGPmfXdEN3tcGnym/OBuR84VScO9EVoMjh81iNU6d5X0Zp562IhmZYyUTotiZ+e1JzFr3McXFu4cNtxvsuW0tpotIoim/73f+q6HWl1fq2norDDJXo55YDYgrctdoLuaWVsTT1IGwc8E9e7fv7lQmhtP0NOaW4JZ/Hwmvl8jagp3LYe4vsQvqTExvJPlNBYzlB6N5RttstDYo6fzfDavqXJzQ2OIMuo68d189k+EwyDINaYBGT5LGxgbM222Adt51y1aXVqTdTdN1+/qict+O7vHWaIi9RBWXwQ1LI2e5pqd5dHDF4XR3PVsfMGyR/Q1zmEegSbdA0KtFJuFfO7iXjgzfpTsueP/D+DH7xavN9I2dNrotcVe6L47Y/lzKnJ0AiIvzlRERAREQEREBERAREQEREBERAVH8XdKy4bUb87Cwux2SLGzuA37GwGhgJ9DXtawA93M3bfd7QrwXjcpwZCrNVtQx2a0zDHLDK0OY9pGxa4HoQR5iu7Q9Kq0O1i0jGN0x1K5PzuNtZWgYKeWs4abmDvCqkcT37DzbSse3Y/Ruo8NE6gAcDxBzh3GwJp4/p8/wD8Mr4z3A2zBM6TT2SjFcnpRyXM4M+Zsw3dt/8Ac1x+daB3CfWTTt4Hi3Hzlt9236t4gf8AJfa06dodtGttLvvMJq5KwxOlMzj8jDYta0y2UgYTzVLNak2OToR1McDXDbv6OHcs/GaNwGEt2LWOweNoWbIInmq1I43y79/M5oBdv86n/wAFGsvkON9vd92nwUay+Q432933a2RpWiR/cj7zf4pqyhEenMTFVx9ZmLpsr49wfThbXYGVnAFoMY22YQHOAI26EjzrDtaE01eyUmQs6exVi/IWufalpROlcWkFpLy3ckEAjr0ICsP4KNZfIcb7e77tPgo1l8hxvt7vu1Z0vRJwmunjBqyqibRefkme9uv83E1ziRG2pjyGj0DesT0+cr495Gof+8POex4//wDmVtfBRrL5Djfb3fdr6j4S6ykcB4LiYh53SX37D6oisPWNE/y/7TzNWUOpxPo0IY7Fp9qSKMNktTBrXSEDq9waA0E952AHoAVu8FdJy1WW9RXIjFJcY2Cmx42cK48ovIPdzu839ljD519aX4JwVJ47WoLbMrIzYilFHyVgd993A7uk2+cgelqs9eF6S9J0WtGwsJvid8+ULuERF8uCIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIP/Z",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import Image, display\n",
    "\n",
    "try:\n",
    "    display(Image(graph.get_graph().draw_mermaid_png()))\n",
    "except Exception:\n",
    "    # This requires some extra dependencies and is optional\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "70912068-e192-4c54-85dd-9422de5f63ae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:53.219925Z",
     "iopub.status.busy": "2024-10-01T03:04:53.219369Z",
     "iopub.status.idle": "2024-10-01T03:04:55.123683Z",
     "shell.execute_reply": "2024-10-01T03:04:55.121993Z",
     "shell.execute_reply.started": "2024-10-01T03:04:53.219876Z"
    }
   },
   "outputs": [],
   "source": [
    "# states\n",
    "results = graph.invoke(input={\"messages\": [(\"human\", \"what's the weather in SF?\")]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9e053126-3e8d-4a21-970b-9242f56fa9ec",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:04:56.137581Z",
     "iopub.status.busy": "2024-10-01T03:04:56.137002Z",
     "iopub.status.idle": "2024-10-01T03:04:56.198378Z",
     "shell.execute_reply": "2024-10-01T03:04:56.196412Z",
     "shell.execute_reply.started": "2024-10-01T03:04:56.137532Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #008000; text-decoration-color: #008000\">'messages'</span>: <span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">HumanMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">\"what's the weather in SF?\"</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'be07a97c-33bb-4603-bf5e-35b0acad09c8'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">AIMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">''</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'tool_calls'</span>: <span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_n7Lp6RmyrMrLVi3KKSeM3nI8'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'arguments'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'{\"city\":\"sf\"}'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'refusal'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'token_usage'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">12</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'prompt_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">122</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">134</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens_details'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'model_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'gpt-3.5-turbo-0125'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'system_fingerprint'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'finish_reason'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'stop'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'logprobs'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'run-cab85c22-7942-42b8-bbf5-7de4f1f2620c-0'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_calls</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'args'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'city'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'sf'</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_n7Lp6RmyrMrLVi3KKSeM3nI8'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'tool_call'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage_metadata</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'input_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">122</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'output_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">12</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">134</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">ToolMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'74038e8c-0a5f-42f2-b7d9-b01902254927'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_call_id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'call_n7Lp6RmyrMrLVi3KKSeM3nI8'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">AIMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">''</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'tool_calls'</span>: <span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_v31rik0EQKhhQFyucrHCO0qg'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'arguments'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'{\"temperature\":75,\"wind_directon\":\"SE\",\"wind_speed\":3}'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'WeatherResponse'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'refusal'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'token_usage'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">24</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'prompt_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">164</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">188</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens_details'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'model_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'gpt-3.5-turbo-0125'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'system_fingerprint'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'finish_reason'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'stop'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'logprobs'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'run-476ed669-9634-4577-898f-286bb7886bce-0'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_calls</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'WeatherResponse'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'args'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'temperature'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">75</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'wind_directon'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'SE'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'wind_speed'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_v31rik0EQKhhQFyucrHCO0qg'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'tool_call'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage_metadata</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'input_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">164</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'output_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">24</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">188</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #008000; text-decoration-color: #008000\">'final_response'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">WeatherResponse</span><span style=\"font-weight: bold\">(</span><span style=\"color: #808000; text-decoration-color: #808000\">temperature</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">75.0</span>, <span style=\"color: #808000; text-decoration-color: #808000\">wind_directon</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'SE'</span>, <span style=\"color: #808000; text-decoration-color: #808000\">wind_speed</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.0</span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[32m'messages'\u001b[0m: \u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mHumanMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m\"what\u001b[0m\u001b[32m's the weather in SF?\"\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'be07a97c-33bb-4603-bf5e-35b0acad09c8'\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mAIMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m''\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'tool_calls'\u001b[0m: \u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_n7Lp6RmyrMrLVi3KKSeM3nI8'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'function'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'arguments'\u001b[0m: \u001b[32m'\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"city\":\"sf\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m'\u001b[0m, \u001b[32m'name'\u001b[0m: \u001b[32m'get_weather'\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'function'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'refusal'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'token_usage'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens'\u001b[0m: \u001b[1;36m12\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'prompt_tokens'\u001b[0m: \u001b[1;36m122\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m134\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens_details'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'model_name'\u001b[0m: \u001b[32m'gpt-3.5-turbo-0125'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'system_fingerprint'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'finish_reason'\u001b[0m: \u001b[32m'stop'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'logprobs'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'run-cab85c22-7942-42b8-bbf5-7de4f1f2620c-0'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mtool_calls\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'name'\u001b[0m: \u001b[32m'get_weather'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'args'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'city'\u001b[0m: \u001b[32m'sf'\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_n7Lp6RmyrMrLVi3KKSeM3nI8'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'tool_call'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33musage_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'input_tokens'\u001b[0m: \u001b[1;36m122\u001b[0m, \u001b[32m'output_tokens'\u001b[0m: \u001b[1;36m12\u001b[0m, \u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m134\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mToolMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m'It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mname\u001b[0m=\u001b[32m'get_weather'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'74038e8c-0a5f-42f2-b7d9-b01902254927'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mtool_call_id\u001b[0m=\u001b[32m'call_n7Lp6RmyrMrLVi3KKSeM3nI8'\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mAIMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m''\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'tool_calls'\u001b[0m: \u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_v31rik0EQKhhQFyucrHCO0qg'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'function'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[32m'arguments'\u001b[0m: \u001b[32m'\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"temperature\":75,\"wind_directon\":\"SE\",\"wind_speed\":3\u001b[0m\u001b[32m}\u001b[0m\u001b[32m'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[32m'name'\u001b[0m: \u001b[32m'WeatherResponse'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'function'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'refusal'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'token_usage'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens'\u001b[0m: \u001b[1;36m24\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'prompt_tokens'\u001b[0m: \u001b[1;36m164\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m188\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens_details'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'model_name'\u001b[0m: \u001b[32m'gpt-3.5-turbo-0125'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'system_fingerprint'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'finish_reason'\u001b[0m: \u001b[32m'stop'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'logprobs'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'run-476ed669-9634-4577-898f-286bb7886bce-0'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mtool_calls\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'name'\u001b[0m: \u001b[32m'WeatherResponse'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'args'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'temperature'\u001b[0m: \u001b[1;36m75\u001b[0m, \u001b[32m'wind_directon'\u001b[0m: \u001b[32m'SE'\u001b[0m, \u001b[32m'wind_speed'\u001b[0m: \u001b[1;36m3\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_v31rik0EQKhhQFyucrHCO0qg'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'tool_call'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33musage_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'input_tokens'\u001b[0m: \u001b[1;36m164\u001b[0m, \u001b[32m'output_tokens'\u001b[0m: \u001b[1;36m24\u001b[0m, \u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m188\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[32m'final_response'\u001b[0m: \u001b[1;35mWeatherResponse\u001b[0m\u001b[1m(\u001b[0m\u001b[33mtemperature\u001b[0m=\u001b[1;36m75\u001b[0m\u001b[1;36m.0\u001b[0m, \u001b[33mwind_directon\u001b[0m=\u001b[32m'SE'\u001b[0m, \u001b[33mwind_speed\u001b[0m=\u001b[1;36m3\u001b[0m\u001b[1;36m.0\u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from rich.pretty import pprint\n",
    "pprint(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "ae34b19a-e7a4-4226-9a72-38af46716dd3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:16.188598Z",
     "iopub.status.busy": "2024-10-01T03:05:16.187973Z",
     "iopub.status.idle": "2024-10-01T03:05:16.200161Z",
     "shell.execute_reply": "2024-10-01T03:05:16.198260Z",
     "shell.execute_reply.started": "2024-10-01T03:05:16.188548Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'name': 'WeatherResponse',\n",
       "  'args': {'temperature': 75, 'wind_directon': 'SE', 'wind_speed': 3},\n",
       "  'id': 'call_v31rik0EQKhhQFyucrHCO0qg',\n",
       "  'type': 'tool_call'}]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][-1].tool_calls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "34b08aa4-8b66-46e7-baf0-531a46514dfd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:17.287647Z",
     "iopub.status.busy": "2024-10-01T03:05:17.287077Z",
     "iopub.status.idle": "2024-10-01T03:05:17.298515Z",
     "shell.execute_reply": "2024-10-01T03:05:17.296859Z",
     "shell.execute_reply.started": "2024-10-01T03:05:17.287599Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "HumanMessage(content=\"what's the weather in SF?\", additional_kwargs={}, response_metadata={}, id='be07a97c-33bb-4603-bf5e-35b0acad09c8')"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "1dc0b200-7f33-46ca-9eae-f5814289a879",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:19.369501Z",
     "iopub.status.busy": "2024-10-01T03:05:19.368924Z",
     "iopub.status.idle": "2024-10-01T03:05:19.380595Z",
     "shell.execute_reply": "2024-10-01T03:05:19.378346Z",
     "shell.execute_reply.started": "2024-10-01T03:05:19.369452Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_n7Lp6RmyrMrLVi3KKSeM3nI8', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 12, 'prompt_tokens': 122, 'total_tokens': 134, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-cab85c22-7942-42b8-bbf5-7de4f1f2620c-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_n7Lp6RmyrMrLVi3KKSeM3nI8', 'type': 'tool_call'}], usage_metadata={'input_tokens': 122, 'output_tokens': 12, 'total_tokens': 134})"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "60e7c88a-99af-4664-99fa-ea31d3286139",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:20.732358Z",
     "iopub.status.busy": "2024-10-01T03:05:20.731774Z",
     "iopub.status.idle": "2024-10-01T03:05:20.743876Z",
     "shell.execute_reply": "2024-10-01T03:05:20.742087Z",
     "shell.execute_reply.started": "2024-10-01T03:05:20.732309Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'id': 'call_n7Lp6RmyrMrLVi3KKSeM3nI8',\n",
       "  'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'},\n",
       "  'type': 'function'}]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][1].additional_kwargs['tool_calls']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "a0be8e51-bb57-477d-aa81-fe13557c9618",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:22.300538Z",
     "iopub.status.busy": "2024-10-01T03:05:22.299965Z",
     "iopub.status.idle": "2024-10-01T03:05:22.311518Z",
     "shell.execute_reply": "2024-10-01T03:05:22.309740Z",
     "shell.execute_reply.started": "2024-10-01T03:05:22.300490Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ToolMessage(content='It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction', name='get_weather', id='74038e8c-0a5f-42f2-b7d9-b01902254927', tool_call_id='call_n7Lp6RmyrMrLVi3KKSeM3nI8')"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "f1022729-b036-4de6-8a94-4cac9c37eb3d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:23.891808Z",
     "iopub.status.busy": "2024-10-01T03:05:23.891253Z",
     "iopub.status.idle": "2024-10-01T03:05:23.903089Z",
     "shell.execute_reply": "2024-10-01T03:05:23.901146Z",
     "shell.execute_reply.started": "2024-10-01T03:05:23.891761Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_v31rik0EQKhhQFyucrHCO0qg', 'function': {'arguments': '{\"temperature\":75,\"wind_directon\":\"SE\",\"wind_speed\":3}', 'name': 'WeatherResponse'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 24, 'prompt_tokens': 164, 'total_tokens': 188, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-476ed669-9634-4577-898f-286bb7886bce-0', tool_calls=[{'name': 'WeatherResponse', 'args': {'temperature': 75, 'wind_directon': 'SE', 'wind_speed': 3}, 'id': 'call_v31rik0EQKhhQFyucrHCO0qg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 164, 'output_tokens': 24, 'total_tokens': 188})"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['messages'][3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2232355-727e-4463-94cf-df4ff1e92a87",
   "metadata": {},
   "source": [
    "### Option 2: 2 LLMs¶"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "842d8847-68c8-472f-9955-2b0b0d4ad92a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:25.700540Z",
     "iopub.status.busy": "2024-10-01T03:05:25.699970Z",
     "iopub.status.idle": "2024-10-01T03:05:25.719772Z",
     "shell.execute_reply": "2024-10-01T03:05:25.717874Z",
     "shell.execute_reply.started": "2024-10-01T03:05:25.700492Z"
    }
   },
   "outputs": [],
   "source": [
    "from langgraph.graph import StateGraph, END\n",
    "from langgraph.prebuilt import ToolNode\n",
    "from langchain_core.messages import HumanMessage\n",
    "\n",
    "tools = [get_weather]\n",
    "llm_with_tools = llm.bind_tools(tools)\n",
    "llm_with_structured_output = llm.with_structured_output(WeatherResponse, strict=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "723e6d1b-e6d6-4e4b-a21c-920a7735d8e5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:26.713990Z",
     "iopub.status.busy": "2024-10-01T03:05:26.713438Z",
     "iopub.status.idle": "2024-10-01T03:05:26.722687Z",
     "shell.execute_reply": "2024-10-01T03:05:26.721114Z",
     "shell.execute_reply.started": "2024-10-01T03:05:26.713943Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that calls the model\n",
    "def call_model(state: AgentState):\n",
    "    response = llm_with_tools.invoke(state['messages'])\n",
    "    # We return a list, because this will get added to the existing list\n",
    "    return {\"messages\": [response]}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "6f1f0dd6-e99c-4eac-a32d-2f87c9f83206",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:28.117767Z",
     "iopub.status.busy": "2024-10-01T03:05:28.117191Z",
     "iopub.status.idle": "2024-10-01T03:05:28.126906Z",
     "shell.execute_reply": "2024-10-01T03:05:28.124981Z",
     "shell.execute_reply.started": "2024-10-01T03:05:28.117718Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that responds to the user\n",
    "def respond(state: AgentState):\n",
    "    # We call the model with structured output in order to return the same format to the user every time\n",
    "    # state['messages'][-2] is the last ToolMessage in the convo, which we convert to a HumanMessage for the model to use\n",
    "    # We could also pass the entire chat history, but this saves tokens since all we care to structure is the output of the tool\n",
    "    response = llm_with_structured_output.invoke([HumanMessage(content=state['messages'][-2].content)])\n",
    "    # We return the final answer\n",
    "    return {\"final_response\": response}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "51cec2d6-10a9-4d15-a245-b8df2b9473fb",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:29.492676Z",
     "iopub.status.busy": "2024-10-01T03:05:29.492110Z",
     "iopub.status.idle": "2024-10-01T03:05:29.502558Z",
     "shell.execute_reply": "2024-10-01T03:05:29.500349Z",
     "shell.execute_reply.started": "2024-10-01T03:05:29.492627Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define the function that determines whether to continue or not\n",
    "def should_continue(state: AgentState):\n",
    "    messages = state[\"messages\"]\n",
    "    last_message = messages[-1]\n",
    "    # If there is no function call, then we respond to the user\n",
    "    if not last_message.tool_calls:\n",
    "        return \"respond\"\n",
    "    # Otherwise if there is, we continue\n",
    "    else:\n",
    "        return \"continue\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "921afd0e-8385-4282-b223-6029ec03088c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:31.033766Z",
     "iopub.status.busy": "2024-10-01T03:05:31.033195Z",
     "iopub.status.idle": "2024-10-01T03:05:31.048425Z",
     "shell.execute_reply": "2024-10-01T03:05:31.046400Z",
     "shell.execute_reply.started": "2024-10-01T03:05:31.033719Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define a new graph\n",
    "workflow = StateGraph(AgentState)\n",
    "\n",
    "# Define the two nodes we will cycle between\n",
    "workflow.add_node(\"agent\", call_model)\n",
    "workflow.add_node(\"respond\", respond)\n",
    "workflow.add_node(\"tools\", ToolNode(tools))\n",
    "\n",
    "# Set the entrypoint as `agent`\n",
    "# This means that this node is the first one called\n",
    "workflow.set_entry_point(\"agent\")\n",
    "\n",
    "# We now add a conditional edge\n",
    "workflow.add_conditional_edges(\n",
    "    \"agent\",\n",
    "    should_continue,\n",
    "    {\n",
    "        \"continue\": \"tools\",\n",
    "        \"respond\": \"respond\",\n",
    "    },\n",
    ")\n",
    "\n",
    "workflow.add_edge(\"tools\", \"agent\")\n",
    "workflow.add_edge(\"respond\", END)\n",
    "graph = workflow.compile()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "c63b8c8a-ee55-4170-b50d-e0b22d6b0282",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:33.227769Z",
     "iopub.status.busy": "2024-10-01T03:05:33.227204Z",
     "iopub.status.idle": "2024-10-01T03:05:34.007757Z",
     "shell.execute_reply": "2024-10-01T03:05:34.006273Z",
     "shell.execute_reply.started": "2024-10-01T03:05:33.227719Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/4gHYSUNDX1BST0ZJTEUAAQEAAAHIAAAAAAQwAABtbnRyUkdCIFhZWiAH4AABAAEAAAAAAABhY3NwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAA9tYAAQAAAADTLQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlkZXNjAAAA8AAAACRyWFlaAAABFAAAABRnWFlaAAABKAAAABRiWFlaAAABPAAAABR3dHB0AAABUAAAABRyVFJDAAABZAAAAChnVFJDAAABZAAAAChiVFJDAAABZAAAAChjcHJ0AAABjAAAADxtbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAHMAUgBHAEJYWVogAAAAAAAAb6IAADj1AAADkFhZWiAAAAAAAABimQAAt4UAABjaWFlaIAAAAAAAACSgAAAPhAAAts9YWVogAAAAAAAA9tYAAQAAAADTLXBhcmEAAAAAAAQAAAACZmYAAPKnAAANWQAAE9AAAApbAAAAAAAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAACAAAAAcAEcAbwBvAGcAbABlACAASQBuAGMALgAgADIAMAAxADb/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCAFlANkDASIAAhEBAxEB/8QAHQABAAIDAQEBAQAAAAAAAAAAAAYHBAUIAwIBCf/EAFQQAAEEAQIDAgcKCggCCAcAAAEAAgMEBQYRBxIhEzEUFSJBUVaUCBYXNlRhcZHR0yMyU1V0gZWytNI1QlJ1k6GxsyY3CRgkV2JzgsE0Q2NykqLU/8QAGwEBAQADAQEBAAAAAAAAAAAAAAECAwQFBgf/xAA4EQEAAQEEBgYJAwUBAAAAAAAAAQIDESFRBBITMZHRM2FxobHBBRQjQUJSYoGSFdLwMkNTouHx/9oADAMBAAIRAxEAPwD+qaIiAiIgIiICIiAiIgIiIC+JZWQRl8j2xsb3ucdgP1rU5rL2WWo8Xi2MkykzO07SZpdDVj327SQAgnruGsBBeQeoAc5uJFoDFTSixlY3Z+51Pb5PaUN36eRHtyMG3TyWj599yVuiimIvrm7xW7NsXaowzCQ7LUQR5jZZ9q/PfVhfzxQ9pZ9q/BpTCNAAw9AAdABVZ9i/ferhfzPQ9mZ9iy9j19y4Hvqwv54oe0s+1PfVhfzxQ9pZ9qe9XC/meh7Mz7E96uF/M9D2Zn2J7Hr7jA99WF/PFD2ln2p76sL+eKHtLPtT3q4X8z0PZmfYnvVwv5noezM+xPY9fcYHvqwv54oe0s+1ZFTM4++/kq3q1l39mGZrj/kVj+9XC/meh7Mz7Fj29D6dvx8ljBY2UebmqM3HXfodtx167hPY9fcmDeIos7GXdHtNjGy28limDeXGTPM8sbf7UD3HmO35NxO46M5duV0jqW4b9WGzXkbNBMwPjkYdw5pG4IWFdGrjTN8Fz2REWpBERAREQEREBERAREQEREBERBF9AbX8bazb9nT5azJPzeiFriyFvzARtadh05nOPeSTKFGOGw7DR9Ok7cS490lGQEbEOikczf6CGgj0gg+dSddGkdLVHX3e7uWd4tNrDWOG0Bpy7n9QZCPGYim0OnsygkN3cGtAABLiXOAAAJJIAG5W5UD45YjEZzhhmKecwuYz2OeYS+pp+Nz77XCZhZLCGkO5o3BsnTr5B6O7jzoiOuvdUaY0vpnTecxsV7L0svnocM8jG3GSV9yO1cYuwMnO1pBbGWgvJ8nfYhSPVXuhtBaIr4ybOZezj2ZGo2/CJMVcLmQHukmaIiYR379qG7bHfbZUjbPEPUHCjF5TMYjUOfh0xr2lkqHhuNEOavYiB7CZJKrQC6UF8g25WueGb8oJ67LitmNRa81PLHZxHEKPSGQ0+PEuNwFWek6fIOfKyVt9zS10OzRDytmc2Mtc4nc7hBc+quOeh9GW8VVymbAs5am6/j4adWe265A0s3dEIWP5z5bTyt3JG7gNgSI9pD3RmG1bxg1DoRlHI15ce2r4LafjbgbYdJFJLJ2hMIbAGhgDS9w5yTyk9yrnghpPNQar4H2clgMnTbhuHtnGW5LtKSMVLbJKkfZuLhs1xEcnL/aaCRuOqmeFsZDQ/umNbvvaezVnG6tr4nxflaFF9ipG6BkscrZ5GjaEguafK2BBQXgiIgKL6V2xmd1BhW7CCCWO9XYN/IjnDiW/4rJiPQHAeZShRjCDwvXOprrd+zihqY7fbYF8YkldsfP0stH0ghdFn/RXHV5x/wBWN0pOiIudBERAREQEREBERAREQEREBERBG78MumcrZy9eF8+Pt8rshBC1z5GPaA0TsaPxvJAa5oG5DGkdQQ781DpDSXFLEVG5vEYnVOMY/tq/hkEdqIO2LeZu4I37xuPnUlUfvaGxVu5LchbPjLspLpLGNsPrulcRtzPDCGvO23VwJ6D0Bb4qori6vCc/5/Opd+9Fv+rXwn/7t9LfsiD+VbrSfB/Q2g8m7I6b0hhMFfdEYXWsdQigkLCQS3ma0HYloO3zBe50TP5tUZ5o9HbRH/WJPeTY9as9/jQ/dK7Oz+fukujNKEUX95Nj1qz3+ND90onxDx+V0xXwD6WqcwXXs1ToS9tLCfwUsnK/b8GPK27v9E2dn8/dJdGa1Fi5TF083jbeOyFWG7QtxOgsVrDA+OWNwIcxzT0IIJBB9K0PvJsetWe/xofuk95Nj1qz3+ND90mzs/n7pLozaCP3N/CmGRr2cONLse0hzXNxMAII7iDyr7qe524W0LcNqtw80zXswvbJFLHioWuY4HcOBDehBG+63nvJsetWe/xoful+nQjJhy287nLce2xYbxhDh85iDD/mmpZx8fdJdGbLzOoxBZOLxnZXc49u7a5d5MDT3STEfiM9Hndts3frtl4HDR4HGMqse6Z/M6WWZ/40sj3Fz3n6XEnbzd3mX3iMJQwNTwbHVIqcHMXFsTduZx73OPeSfOT1KzljVXF2pRu8TsERFpQREQEREBERAREQEREBERAREQEREBERAVe8ZSBU0huSP+KMb3f+d9KsJV7xl38E0htt8Z8b3gflfnQWEiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgKvOMw3p6P6hv/FGN7x/9ZWGq84z7eB6P39aMZ5t//nILDREQEREBERAREQEREBERAREQEREBERAREQEREBEUd1BqexSujHYqnHfyXIJZBPKYoYGEkNL3BrjuSDs0A77HcgbE7KKKrSdWlbr0iRQjx7rD5Bg/a5vu08e6w+QYP2ub7tdHqteccYLk3XF3uyfde2uDWv8AEaWvaFku1KtulnKeUGSaxtxkbt3M5DC7kIeHN33PcD59l01491h8gwftc33aqP3QXAW17oqHTTNQUsRA7C322myQWpeaaE7drAT2fRr9m9fNsnqteccYLlw8Htd3eJ/DPT+q8hg36bsZev4U3GyT9u6KNzj2RL+Vu/Mzkf3Dbm267bqZKCwZbVlaGOGHGYGKKNoYyNlmYNa0DYAARdAF9+PdYfIMH7XN92nqteccYLk3RQjx7rD5Bg/a5vu19M1Jqmse0s4jG2oW9Xx07jxMR5+QPjDXH0AuaD6QnqtpnHGC5NUWLjMlWzGPr3qkna1rDBJG/YgkH0g9QfSD1B6FZS5JiYm6UERFAREQEREBERAREQEREBERAREQFA6x315qjfzCoP1dmftKnigdX4+ao+ir/tFdui/H2ecMo3S3KIi3MRERARFg5fOY/AwQzZK7BRimnjqxvsSBgfLI4Mjjbv3uc4gADqSUGciIgx+Fx30ZW+azbA+YCzKApWonwu+Jlf8ASrn8VKpYuXSentO2fFZ3yIiLmQREQEREBERAREQEREBERAREQFA6vx81R9FX/aKnigdX4+ao+ir/ALRXbovx9nnDKN0tyudeP93Kal1fewmlbOpW5rC4Txjbkx2oTiaNNr3Sdk94bG8zykxv8gjk5W9SN10UoXq/g1o7Xuchy+dwrb1+ODwUyCxLE2aHmLhFMxjw2Zm5J5JA5vU9OpWyqL4Yue8nxIv6nqcMbOtNT6g01p7M6KZkm3dNPlhkuZgiMva4wtLujHc7I/xXFxBB22WvxF7idnZuHXD+4chDkWaRGdyMMmpJ8RbtWHWDGe0sNimlJY3lLohygGXr0aGqzOJXud5bcWmqmjcJhfFuHqS044shnMrQmhjc8PDWTVpCXRgj8R4O3TlLQNlIcH7n3GZXh5pjC8QJX6tzmFEro8y21YgsxF73HkjnbIJuUNLY+r93BgLuq16s3iuvE/ECDUXCvSGr9UZCochezLJ34bLvdNYpxwCWCOawI4i+Ru3KZAxrthuCC4lRXWFS1qLh/YwmXzmauwad4rU8PUuPyUzbPgrp6zmiSVrg57mCdwa9xLmlrSDuAV0/jOFelsO/TLqeKbA7TYnGL5ZpP+z9s0tlPV3llwJ3L+Y7nfv6rxv8INIZTBaiw1vDR2MbqC6clkoJJpD21kiMdqDzbscOyjI5C3YtBGx6q6siSYTExYHE1MfDPasxVoxG2W7ZfYmeB53yPJc8/O4krNWr0zpuhpDB1cRi45YqNYOEbZ7Elh43cXHeSRznu6k95K2i2DG4XfEyv+lXP4qVSxRPhd8TK/6Vc/ipVLFzaT09p2z4rO+RERcyCIiAiIgIiICIiAiIgIiICIiAoHV+PmqPoq/7RUvzOaoaex0t/JW4qVOMta6aZ3KOZzg1jR6XOc5rQB1JcAASQFEbdPNjN5TMVMK4skkFV1OWyxsthkY8mxGdy1vMXObyOIJa1jt2ndp69GqiJqpmd8Xd8T5LDcItJ42z3qZlfaqX36eNs96mZX2ql9+uzU+qPyjmtzdotJ42z3qZlfaqX36w8nqzLYhtZ1nRuZAsTsrR9nJVk8t52bvyzHlG/e47AechNT6o/KOZck6LSeNs96mZX2ql9+njbPepmV9qpffpqfVH5RzLm7RRnMary2CxN3JXNH5dlSnA+xM5k1SRwYxpc4hrZiXHYHoASfMFWnDr3XGieL+QZi9K367MnM9sUUeYsR1d3O6DlY53PId/6rRue7cJqfVH5RzS5dHC74mV/wBKufxUqli1uncKzT2Fq4+OR0whaeaV/e9xJc5x+lxJ/WtkvOtqortaq43TM+JO8REWlBERAREQEREBERAREQERY+QyFXE0LN69Zhp0q0bpp7NiQMjijaCXPc49GtABJJ6ABBkLVX86IL1ejUqzX7Mz3xudEPwNYtj5953/ANQHmYAOrjzghpaHEYRnyOpy9tV0+IxX/Zp4MjG5hmtsPlyMEb2nsmkcjC53l+VJsGFrXnbYvD0cJWfXx9OCjC+WSd8deMMDpJHF8jzt3uc5znOcepJJO5KDCw+CmhmbkMnaNzLSVooZ+yc9lVrm7uJhhc5wZu4k7klxAaC48rdtyiICIiAo5rWZ8MOHLJcnFzZWq0+LIucuBf1bL6Ij/XPmCkajutp/BqmLf2+SgBylRm+MZzudzStbyyDzRHfyz5m7lBIkREHy9jZWOY9oexw2LXDcEegri7hR7gZnB7iXluIMsWO1bNRy77OCwDido6m5IkEj+VguNJBjDwYgY+r2ukbNB2miDW6e1Fj9VYiDJ4ux4RTm5gC5jo3se1xa+N7HAOjkY5rmOY8BzHNc1wBBA2Sh+ocBkMPl5dS6bY6a49oGRw3OGx5RjW7NcwuIbHZaAA2QkNe0COQ7COSHe6c1HQ1XiYsjjZjLXeXMLXsMckT2ktfHIxwDmPa4FrmOALSCCAQg2aIiAiIgIiICIiAiIgIi1uYyr6PZVqsQsZK02TwWJ/MIy5rC7eR7WuMbNw1pdsdi5o2JIBD1yuYrYeOu6z2pNixHWiZDC+V7nvdsPJYCdh1c53c1rXOcQ1pIwMfirl6zWyOYcI7cInZHSqzuNdrHu8kvB255AxrRuRsC54b0JJyMRg24+xYvTyeEZS5HCy1YbztY4xt2AjY5zuzZuXuDQT1e4kkklbRAREQFjy368DyySZrHjvBKyFFM7/Skv0N/0CCQeNanyhn1p41qfKGfWqVw3HDROodUe9/GZrw7JGWSBphqzurvkYCXsbY5Oyc5oa7cB5PQr503x00Lq/UUeExGfjt5CYyNrjsJWRWjHv2ghmcwRzcuxJ7NztgCfMguzxrU+UM+taPWN+GXEQmHIXKz479OQuxrQ6VzW2Yy5hH5NzQWv9DC4juVU4vj/oHNZanjqeeEti3adQheak7YXWWuc0wGV0YY2Tdp2YXBx6EAgjfUac90Jh9Qax1xp+WpkajtOTvibPHjbconZHA2WV+4h5WkEua1m5LwAW8wc1B0T41qfKGfWnjWp8oZ9a53017oDTzNF6Zyeos5SfezzbclIYihcc222GbkcIonRmUvaHR7sLeYnmLQWgkSnF8W9JZqpp+zSzDLEOetSUceWwyAyTxte58bgW7xuaIn7h4bsW7d+wQW/wCNanyhn1p41qfKGfWqlyPFLSuIOpPDsxFUGnHQx5R0zHtEDpWNfE0EjZ5c17dgzmO7gO/ov3RHFDTPEU3GYHJG1PT5fCas9eWtPCHAlpdFK1rwDsdjtsdjseiC4YpWTsD43B7D3EKJ6lwd/DZGfU2nIDYyBYPD8SHBrcpGwdOUuIayw0dGPJDXDZjyByPi3+C/ouH/ANX+pWwQa7T2foapw1TK4yfwilZZzMcWOY4ddnNexwDmPaQWuY4BzXAggEELYqvbB94HEuk6P8HgtWyOgkia0BlfJsjdI2QegTxRva7fpzwx7eVId7CQEREBERAREQEREGBmso3E0HygwusPPZ1oZ52wieY/iRhx7i49O4n5ivPCYo0GT2J9zkLrmzWj275WNfyNbyx8/wCLGOXo1oaNy5xHM5xOtsujy2va1N8uMsR4imL7qkkXaW4J5nPihna49I2ljLbNx1dzOG4AIMlQEREBERAUI1rTlyDMnVgmNaaeu6Jkw743OZsHfqJ3U3Udy+Ns2L8kkcRcw7bEEegIOYOFORydLhRjeFNvR+osDn6+Klw82Qbj3eLo5BC9vhIsg8jmvds7yd3cz+o7ytJp2nnNT4Tg1oqLRmawOR0ffpWcvevUjDTgZVgfG8Qz/izGZzthyF3RxLtl1d4mufkHfWE8TXPyDvrCDlOno/Ox+5503QOEyLcpX1wy66r4JIJo4hnHydqW7bhvZnn5ttuU777dVO9DT3tI8cOIGOv4LLmtqPIVshQytek+SkY20o43iSZvkxuD4XDZ2xO7du9Xj4mufkHfWE8TXPyDvrCDlLg7o/O4y5wNdcwmRqjG1NRtuusVJGeCulmYYhJuPILwCW77cw7t14xaazunr8WoZNPZaxTxHE3KZKatVpSSWHU54JY22IogOaRnNID5AO4JI32K6tdi7cVhjDXmJl32IG7WkDznzb/OvbxNc/IO+sIOMtY6S1Lr7K671HU0vqOvRi1ZhMy3H8slC9fpwUhFIa7g5pDwTzgBwcCwNPK7oLk4MYDT0+dymo8bhdaUch4NHQdc1lPcdLNFzF/JGyzI5wa13UnYDd3Tfcq6fE1z8g76wnia5+Qd9YQb7Bf0XD/6v9StgsPEQvr0I45G8rxvuD9JWYgrzjztDw7feBDZ8dk8behcd9w+O7A7Ybf2gC35w4g9CVYarzjdte0/gsGCe1zOocbWa0DfmZHZZZmH+DXm+jvVhoCIiAiIgIiICIvC7dr4ylYuXLEVSpXjdLNPO8MjjY0buc5x6AAAkk9AAg0uNn59cZ2Lw2jLyVKZ8Eij2sw7un8qV3na7byB5uV/pUhVZYHi7ofIa+yUFbXWkLcluKlXq1ql6A25JS6UcpcHfhNy5oYwbkHm6eUFZqAiIgIiICIiAiIgIiINbn8HX1BjxXsR85jljswntHxlksbw+N3Mwh2wc0bgHqNwdwSD+4DJvyuMilsMhgvMAjt1oJxM2CYAc8fMO/YnzgHYgkDuWxUYyctbSOcblJJaOPxmTljr3D4KRLNde6KGu90renUAReWOv4IBw25XBJ0WpwOq8PqiXLR4jIQZB2Kuvx1013czYbLGsc+InuLmh7QQN9ju07EEDbICIsXK5SphMZcyN+dlWjThfYsTyHZscbGlznE+gAE/qQQe2/3z8bKNZj3GnpbGuuTNH4pt2yYoevpZDFZ3HosMPTpvYShHCTGWo9NTZzJ15KuY1HYdl7cEzgXwdo1rYYDt03ihZDGdunMxx86m6AiIgLS5jW2ntP2hWyecx2Pskc3Y2bTGP29PKTvss3NXHY/D3rTAC+CCSVoPpa0kf6KI6SqR1sBSkA5p7MTJ55ndXzSOaC57iepJJ/V3dwXXY2VNVM117upYzlsvhS0d604j22P7U+FLR3rTiPbY/tXsi3bKxynjHJcHj8KWjvWnEe2x/avK5xH0PkKk9WzqPCz1p2OilikuRlr2OGxaRv1BBIWWibKxynjHIwfzv4Ee5q09or3YGUyGRzWOGiNNT+M8Pckts7O09x5q7Gu3G7oj1dt3GMb/AIwX9CvhS0d604j22P7V7Imyscp4xyMHj8KWjvWnEe2x/anwpaO9acR7bH9q9kTZWOU8Y5GDx+FLR3rTiPbY/tW3w2pMTqJsjsXk6mREe3P4LO2Tk3G432PTcdeq1y0OqOXHvxmVhAjuwX6sDZWjynRzTxxSRn0tIfvsdxu1rtt2ja7CzrnVpviZ67/KDCcFgoiLzWIiIgIiINTmdW4TTsjI8pl6OOke3nay1YZG4t323AJ3236brnr3XOPn4z8NzitD8Wcfpq3EJXWMezIMhiyrHN5exllaedreUvHL1Y7n8sdAW2toLlt6Zo5V4D7uUiZdszkeXI97Q7qfQBs0DuDWgDYABSJenNhZUTNNV8zHXEeUssIwcjf9HRfh4XcONV4DVtypgrsOac5jLdljWyjsmAujdvyvbu38ZpIXWfwpaO9acR7bH9q9kWOyscp4xyMHj8KWjvWnEe2x/aoLxN4g6a1ZZwGlYs/jX4rI2TZzFnwlhibTg5XuhLt9uaaQwx8p6mMzEfiFWAibKxynjHIwePwpaO9acR7bH9qycfxB0xlbUdanqHGWbEjgxkUVuNznuPcAN+p+ZfC8L1Gvkqsla1CyxXkaWvjkbu1wTZWOU8Y5JglSKN8PL82R0jTksSvnliknrGWQ7ueIpnxgkkkk7MG5PU95UkXFaUTZ1zRPum4nBq9VfFjMfoc37hUe018XMV+iRfuBSHVXxYzH6HN+4VHtNfFzFfokX7gXbY9DPb5HubJEVDad90/au8K7HEjO6Sbp/SEdd5imdlWy2bFgTiBkbYzG1oY95IEj3t223LQ3ylZmIRfKLnTG+7IxYZm2ZPGY6W3Qw1nNQRaZ1BXzLZo4Nu0ie6MNMUmzmnygWkBxDjylSK5x61DiuHmN1Lc0TUfNlMrQx2Pq0c/HYitNtPaxsjZxEANi7YgjYkfjbdVjrQLpRUpk/dJO0X75qWtdMS4XPYilBkK1HHXW3mZKKabweLsZOSM8xmLWFrmjbmB3I6r7scf8vptmpamrdFjA5zGadtakpVIMq21Beggb+Ej7YRt7ORriwEcrhs/cFwCutAuhFUWm+O96zqLBUtTaWGmMfqDGT5XF3jkmWSY4WMkkZOxrAInCOQO6OeNgeu4UB1Bxq1bre5wsylHTd3TOjs1quoKmU8ahs+QquinLWzVmtBbHIAHgFzhs0bgdFNaB02tBrf8Aoep/emO/jYFv1oNb/wBD1P70x38bAuix6SnthlTvhYSIi8diIiICIiCueG3/AC90z/dtf/bapGo5w2/5e6Z/u2v/ALbVI17Nv0tfbPis75EXPnCrjVqmvpziTqLX9GpX0/p3K5UG5Vv9vNEK8gAqsiEEYc1rQQ2Qu3cdt2jfcb3G+6Dv4rI4+LXmkH6MpZXHWslj7YyLbnMyvF20sczWsaYpBFu/YcwPK4c24XNrQi5kVN6U4+5XKZTSR1BombTeB1cSzC5J+RZYkc8xOmiZYhDR2LpI2uIAc/qNjsVCuHnHPLaU4H8KzNUl1VqTVE9inDYy+V8GiL2Pmf8AhbMjXnmLWBrG7EuPQdya0DplFhYS3cvYelZyFHxXflhY+el2zZuweRu5nO3o7Y9Nx0OyzVmMXhb8Tov029/FzKWqJcLfidF+m3v4uZS1cuk9Pads+Kzvlq9VfFjMfoc37hUe018XMV+iRfuBSHVXxYzH6HN+4VHtNfFzFfokX7gXRY9DPb5HubJUdR9z5fte5lpcNcjlK9PNVWtlhyNMGaKKxHaNiF4Dg0uaHBoIIG43+lXiisxEorPT2P4neK8ucxU0PVyXgZjx4x4sywvsdd3zFzWERnp5Ddz3+UVRGt+FepuHemZchJHp7FXM5rjT09DA4Z0xxdOdlhjTJ5TWEGV5aXhrB0aNtz1XYaLGabxzzqD3PurOJ9jU+d1hlcPjNSWaFShhY8KJZ61AV7QtsfI6RrHSOfM1nMA1uzRsNydxk53g1rviNNqbL6staeqZebSl/TeIo4mWd9aN9pv4SxNJIwOG5bGOVrTyt36uKv1E1YFQ5rgxez2S4a+FT1DjtP4i9jMmxsjhJL4RUjg3h8jYjdrty7l6bdD3KI4fgxxNix/DfTmUv6Xt6e0Tl6lqC/C+wy7bq14pIow6MsLGyBjxvs4gkb7jz9GImrALQa3/AKHqf3pjv42Bb9aDW/8AQ9T+9Md/GwLosekp7YZU74WEiIvHYiIiAiIgrnht/wAvdM/3bX/22qRqOcNv+Xumf7tr/wC21SNezb9LX2z4rO+VEs4GaitY7iVo6/axMmiNXWshfiuxPlGRqzWtncpiLezc1j9yDz7kbdAvGfgnrPiZk8OOJN7BtxeGxl6jDHgHTOkuy2qxrSTy9q0CPaJz9mN5vKfvzdAFfiLm1YRRGnOEGvchkOH9HWOSwEmntESts1ZMV23hWSnigdBA+Zr2hsPK17nENc/d3oCxcVwb1vp/g9X0CaGiNV42rbsRNhznhDY7FJxL4nO5WO5Jmve7fYOGwBDgT06BRNWBEeEmjr/D7hrp3TmUyjs1kMbUZXmuu5vwhHmHMSeUDZo367NG6lyIstwxeFvxOi/Tb38XMpaolwt+J0X6be/i5lLVzaT09p2z4rO+WHmabsjiL1RhAfPBJECfMXNI/wDdRDSVyOxgacIPJZrQsgsQO6Phka0BzHA9QQfrGxHQhTtaXMaK0/qGwLGUweNyM4HKJbVSOR4Ho3cCdlbG1pppmivcdTGRePwV6M9U8J+z4v5U+CvRnqnhP2fF/Kt+1sc54RzMHsi8fgr0Z6p4T9nxfyp8FejPVPCfs+L+VNrY5zwjmYPZFANPcO9LzcYNaU5NPYqSnBjMU+Gq6nEY4nOdc53Nbt0LuVu52G/KO/bpOfgr0Z6p4T9nxfyptbHOeEczB7IvH4K9GeqeE/Z8X8qfBXoz1Twn7Pi/lTa2Oc8I5mD2Wh1OWZGXGYiFwlvT36tgQtPlNihnjlkkcPM0Bm252G7mt33cN9x8FejPVPCfs+L+VbjD6dxWno5I8XjamNZJtztqQNiDthsN+UDfYK7ezonWpvmY6rvOVwjFsURF5rEREQEREFdaDLKem6WIeQy9iomUrMDj5cb2NDdyOnQjZwPcQ4EbghSJZ2Z0nhNRPa/K4ejkntbyNdbrMlIbvvsC4Hpv12Wr+CvRnqnhP2fF/KvTm3sq5mqq+JnqifOGWE4vZF4/BXoz1Twn7Pi/lT4K9GeqeE/Z8X8qx2tjnPCOaYPZF4/BXoz1Twn7Pi/lT4K9GeqeE/Z8X8qbWxznhHMwey8L1+tjKslm3MyvBGN3SSHYBfvwV6M9U8J+z4v5VlY7h/pjEWmWaOncVTsMIcyWClGx7SO4ggbgptbHOeEczB58PKE2N0lTjsRPglkkmsmKQbOYJZnyAEeYgPG4PUKRoi4rSubSua5983k4iIi1oIiICIiCvtMk/Dbrsc27RisRs3r08q7+r6vR9CsFV9plpHG/Xh5NgcThxz9evl3eno6f+/0KwUBERAREQEREBERAREQEREBERAREQEREBERAREQEREBERBXumWEccNeO5XAHE4ccx7j5d3oPr/zCsJV9pkD4bteHZu/inD9Rvv8AjXf1fV+vzKwUBERAREQEREBERAREQEREBERAREQEREBERARFiZbLVMHjrF+/O2tUgbzySP8AMPo7ySdgAOpJAHUqxE1TdG8ZaKkc7xozeTlc3DVoMRT/AKs1tnbWHj08oIaz6Dz/AKlH3a81i7b/AIqtNP8A4adTb/OEr3rP0LpNcX1TFPbM+USuGbo5eF9tl1GwKT4o7hjcIHzsL42v28kuaCCW77bgEEjzhc7+/rWXrZc9kqfcp7+tZetlz2Sp9ytn6FpHz09/7TDNQ3A73TPGfWXus8hpO/pzTNS/NLDR1AY6lkCrVpvlL3RntyGvcJngF3MCSzp37/0IXJGJxlrB66zOsqOVmr6mzEMde9kG1axfNHHtyjYxcre4b8oG+w332ClHv61l62XPZKn3KfoWkfPT3/tMM3R6LnD39ay9bLnslT7lfTNfayjcHe+ixLt/Vlp1S0//AIxA/wCafoWkfPT38jDN0aiqDTPG2zBK2DU1aAQHp4yotc1rP/MiJJA9LmuPztAG6t1j2yMa9jg5rhuHA7gj0rx9J0S10WrVtY38JR9IiLkBERAREQEREBERAREQEREBERAVDcWNSSZ/V0uNa7fH4gtYGb9H2S0Oc8+nla4NHoJer5XMGU5hqXUQk37QZW3vzHzdq4t//UtX0XoSyprt6q5+GMPuu6HgiIvtmsRRPi1kMpiuGWqLmE5/GsGPmfXdEN3tcGnym/OBuR84VScO9EVoMjh81iNU6d5X0Zp562IhmZYyUTotiZ+e1JzFr3McXFu4cNtxvsuW0tpotIoim/73f+q6HWl1fq2norDDJXo55YDYgrctdoLuaWVsTT1IGwc8E9e7fv7lQmhtP0NOaW4JZ/Hwmvl8jagp3LYe4vsQvqTExvJPlNBYzlB6N5RttstDYo6fzfDavqXJzQ2OIMuo68d189k+EwyDINaYBGT5LGxgbM222Adt51y1aXVqTdTdN1+/qict+O7vHWaIi9RBWXwQ1LI2e5pqd5dHDF4XR3PVsfMGyR/Q1zmEegSbdA0KtFJuFfO7iXjgzfpTsueP/D+DH7xavN9I2dNrotcVe6L47Y/lzKnJ0AiIvzlRERAREQEREBERAREQEREBERAVH8XdKy4bUb87Cwux2SLGzuA37GwGhgJ9DXtawA93M3bfd7QrwXjcpwZCrNVtQx2a0zDHLDK0OY9pGxa4HoQR5iu7Q9Kq0O1i0jGN0x1K5PzuNtZWgYKeWs4abmDvCqkcT37DzbSse3Y/Ruo8NE6gAcDxBzh3GwJp4/p8/wD8Mr4z3A2zBM6TT2SjFcnpRyXM4M+Zsw3dt/8Ac1x+daB3CfWTTt4Hi3Hzlt9236t4gf8AJfa06dodtGttLvvMJq5KwxOlMzj8jDYta0y2UgYTzVLNak2OToR1McDXDbv6OHcs/GaNwGEt2LWOweNoWbIInmq1I43y79/M5oBdv86n/wAFGsvkON9vd92nwUay+Q432933a2RpWiR/cj7zf4pqyhEenMTFVx9ZmLpsr49wfThbXYGVnAFoMY22YQHOAI26EjzrDtaE01eyUmQs6exVi/IWufalpROlcWkFpLy3ckEAjr0ICsP4KNZfIcb7e77tPgo1l8hxvt7vu1Z0vRJwmunjBqyqibRefkme9uv83E1ziRG2pjyGj0DesT0+cr495Gof+8POex4//wDmVtfBRrL5Djfb3fdr6j4S6ykcB4LiYh53SX37D6oisPWNE/y/7TzNWUOpxPo0IY7Fp9qSKMNktTBrXSEDq9waA0E952AHoAVu8FdJy1WW9RXIjFJcY2Cmx42cK48ovIPdzu839ljD519aX4JwVJ47WoLbMrIzYilFHyVgd993A7uk2+cgelqs9eF6S9J0WtGwsJvid8+ULuERF8uCIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiICIiAiIgIiIP/Z",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "try:\n",
    "    display(Image(graph.get_graph().draw_mermaid_png()))\n",
    "except Exception:\n",
    "    # This requires some extra dependencies and is optional\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "8ca0c8f6-3a35-4a90-8ffa-3027694d1d61",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:36.187735Z",
     "iopub.status.busy": "2024-10-01T03:05:36.187156Z",
     "iopub.status.idle": "2024-10-01T03:05:38.945512Z",
     "shell.execute_reply": "2024-10-01T03:05:38.943761Z",
     "shell.execute_reply.started": "2024-10-01T03:05:36.187687Z"
    }
   },
   "outputs": [],
   "source": [
    "results = graph.invoke(input={\"messages\": [(\"human\", \"what's the weather in SF?\")]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "cee24256-6a55-44be-99ef-18f3a3e87530",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:44.314746Z",
     "iopub.status.busy": "2024-10-01T03:05:44.314153Z",
     "iopub.status.idle": "2024-10-01T03:05:44.357304Z",
     "shell.execute_reply": "2024-10-01T03:05:44.355218Z",
     "shell.execute_reply.started": "2024-10-01T03:05:44.314699Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #008000; text-decoration-color: #008000\">'messages'</span>: <span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">HumanMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">\"what's the weather in SF?\"</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'8912de4b-fb1d-42cb-9ac0-78e36e9550a3'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">AIMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">''</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'tool_calls'</span>: <span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_4p8mre75woEdOWCuKYMcCeYV'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'arguments'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'{\"city\":\"sf\"}'</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'refusal'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'token_usage'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">14</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'prompt_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">59</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">73</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens_details'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'model_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'gpt-3.5-turbo-0125'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'system_fingerprint'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'finish_reason'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'tool_calls'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'logprobs'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'run-5c25a00c-8179-40df-85b9-00b0851b80b7-0'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_calls</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'args'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'city'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'sf'</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'call_4p8mre75woEdOWCuKYMcCeYV'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'type'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'tool_call'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage_metadata</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'input_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">59</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'output_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">14</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">73</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">ToolMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'get_weather'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'5e0140fb-cb35-40d6-8390-80fcbc5d5bd7'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_call_id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'call_4p8mre75woEdOWCuKYMcCeYV'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">AIMessage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'The weather in San Francisco is currently 75 degrees and sunny, with 3 mph winds in the South-East direction.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">additional_kwargs</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'refusal'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">response_metadata</span>=<span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'token_usage'</span>: <span style=\"font-weight: bold\">{</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">25</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'prompt_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">101</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">126</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'completion_tokens_details'</span>: <span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'model_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'gpt-3.5-turbo-0125'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'system_fingerprint'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'finish_reason'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'stop'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #008000; text-decoration-color: #008000\">'logprobs'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">}</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'run-10882c55-1f62-4267-ae72-419cf2001ff0-0'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage_metadata</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'input_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">101</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'output_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">25</span>, <span style=\"color: #008000; text-decoration-color: #008000\">'total_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">126</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #008000; text-decoration-color: #008000\">'final_response'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">WeatherResponse</span><span style=\"font-weight: bold\">(</span><span style=\"color: #808000; text-decoration-color: #808000\">temperature</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">75.0</span>, <span style=\"color: #808000; text-decoration-color: #808000\">wind_directon</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'SE'</span>, <span style=\"color: #808000; text-decoration-color: #808000\">wind_speed</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.8</span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[32m'messages'\u001b[0m: \u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mHumanMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m\"what\u001b[0m\u001b[32m's the weather in SF?\"\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'8912de4b-fb1d-42cb-9ac0-78e36e9550a3'\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mAIMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m''\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'tool_calls'\u001b[0m: \u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_4p8mre75woEdOWCuKYMcCeYV'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'function'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'arguments'\u001b[0m: \u001b[32m'\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"city\":\"sf\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m'\u001b[0m, \u001b[32m'name'\u001b[0m: \u001b[32m'get_weather'\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'function'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'refusal'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'token_usage'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens'\u001b[0m: \u001b[1;36m14\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'prompt_tokens'\u001b[0m: \u001b[1;36m59\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m73\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens_details'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'model_name'\u001b[0m: \u001b[32m'gpt-3.5-turbo-0125'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'system_fingerprint'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'finish_reason'\u001b[0m: \u001b[32m'tool_calls'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'logprobs'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'run-5c25a00c-8179-40df-85b9-00b0851b80b7-0'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mtool_calls\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'name'\u001b[0m: \u001b[32m'get_weather'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'args'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'city'\u001b[0m: \u001b[32m'sf'\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'id'\u001b[0m: \u001b[32m'call_4p8mre75woEdOWCuKYMcCeYV'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'type'\u001b[0m: \u001b[32m'tool_call'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33musage_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'input_tokens'\u001b[0m: \u001b[1;36m59\u001b[0m, \u001b[32m'output_tokens'\u001b[0m: \u001b[1;36m14\u001b[0m, \u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m73\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mToolMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m'It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mname\u001b[0m=\u001b[32m'get_weather'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'5e0140fb-cb35-40d6-8390-80fcbc5d5bd7'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mtool_call_id\u001b[0m=\u001b[32m'call_4p8mre75woEdOWCuKYMcCeYV'\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1;35mAIMessage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m'The weather in San Francisco is currently 75 degrees and sunny, with 3 mph winds in the South-East direction.'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33madditional_kwargs\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'refusal'\u001b[0m: \u001b[3;35mNone\u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mresponse_metadata\u001b[0m=\u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'token_usage'\u001b[0m: \u001b[1m{\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens'\u001b[0m: \u001b[1;36m25\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'prompt_tokens'\u001b[0m: \u001b[1;36m101\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m126\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[32m'completion_tokens_details'\u001b[0m: \u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'model_name'\u001b[0m: \u001b[32m'gpt-3.5-turbo-0125'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'system_fingerprint'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'finish_reason'\u001b[0m: \u001b[32m'stop'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[32m'logprobs'\u001b[0m: \u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m}\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'run-10882c55-1f62-4267-ae72-419cf2001ff0-0'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33musage_metadata\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'input_tokens'\u001b[0m: \u001b[1;36m101\u001b[0m, \u001b[32m'output_tokens'\u001b[0m: \u001b[1;36m25\u001b[0m, \u001b[32m'total_tokens'\u001b[0m: \u001b[1;36m126\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[32m'final_response'\u001b[0m: \u001b[1;35mWeatherResponse\u001b[0m\u001b[1m(\u001b[0m\u001b[33mtemperature\u001b[0m=\u001b[1;36m75\u001b[0m\u001b[1;36m.0\u001b[0m, \u001b[33mwind_directon\u001b[0m=\u001b[32m'SE'\u001b[0m, \u001b[33mwind_speed\u001b[0m=\u001b[1;36m4\u001b[0m\u001b[1;36m.8\u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from rich.pretty import pprint\n",
    "pprint(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "040be0e0-74d9-419d-9959-5e6d61438857",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:48.096086Z",
     "iopub.status.busy": "2024-10-01T03:05:48.095517Z",
     "iopub.status.idle": "2024-10-01T03:05:48.107181Z",
     "shell.execute_reply": "2024-10-01T03:05:48.105303Z",
     "shell.execute_reply.started": "2024-10-01T03:05:48.096033Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WeatherResponse(temperature=75.0, wind_directon='SE', wind_speed=4.8)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results['final_response']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "c75cbbf6-89c4-4aa9-a1e2-d5e2be3cc027",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:49.201203Z",
     "iopub.status.busy": "2024-10-01T03:05:49.200645Z",
     "iopub.status.idle": "2024-10-01T03:05:49.212949Z",
     "shell.execute_reply": "2024-10-01T03:05:49.211082Z",
     "shell.execute_reply.started": "2024-10-01T03:05:49.201153Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'messages': [HumanMessage(content=\"what's the weather in SF?\", additional_kwargs={}, response_metadata={}, id='8912de4b-fb1d-42cb-9ac0-78e36e9550a3'),\n",
       "  AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_4p8mre75woEdOWCuKYMcCeYV', 'function': {'arguments': '{\"city\":\"sf\"}', 'name': 'get_weather'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 14, 'prompt_tokens': 59, 'total_tokens': 73, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-5c25a00c-8179-40df-85b9-00b0851b80b7-0', tool_calls=[{'name': 'get_weather', 'args': {'city': 'sf'}, 'id': 'call_4p8mre75woEdOWCuKYMcCeYV', 'type': 'tool_call'}], usage_metadata={'input_tokens': 59, 'output_tokens': 14, 'total_tokens': 73}),\n",
       "  ToolMessage(content='It is 75 degrees and sunny in SF, with 3 mph winds in the South-East direction', name='get_weather', id='5e0140fb-cb35-40d6-8390-80fcbc5d5bd7', tool_call_id='call_4p8mre75woEdOWCuKYMcCeYV'),\n",
       "  AIMessage(content='The weather in San Francisco is currently 75 degrees and sunny, with 3 mph winds in the South-East direction.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 25, 'prompt_tokens': 101, 'total_tokens': 126, 'completion_tokens_details': {'reasoning_tokens': 0}}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-10882c55-1f62-4267-ae72-419cf2001ff0-0', usage_metadata={'input_tokens': 101, 'output_tokens': 25, 'total_tokens': 126})],\n",
       " 'final_response': WeatherResponse(temperature=75.0, wind_directon='SE', wind_speed=4.8)}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10a921cd-7356-4d14-9197-478836143f32",
   "metadata": {},
   "source": [
    "### `strict=True`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab3258b8-80e8-4cb9-b2ba-9c8028c5e3e0",
   "metadata": {},
   "source": [
    "- openai official：https://openai.com/index/introducing-structured-outputs-in-the-api/\n",
    "    - August 6, 2024\n",
    "    - gpt-4o-2024-08-06\n",
    "    - https://platform.openai.com/docs/guides/structured-outputs\n",
    "- langchain 兼容\n",
    "    - https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html\n",
    "    - `llm.bind_tools([GetWeather, GetPopulation], strict=True)`\n",
    "        -  enforce tool args schema is respected\n",
    "    -  `llm.with_structured_output(Response, strict=True)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "0a8f275c-d773-4354-8833-be7c954f0178",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:51.421077Z",
     "iopub.status.busy": "2024-10-01T03:05:51.420525Z",
     "iopub.status.idle": "2024-10-01T03:05:51.430074Z",
     "shell.execute_reply": "2024-10-01T03:05:51.428080Z",
     "shell.execute_reply.started": "2024-10-01T03:05:51.421028Z"
    }
   },
   "outputs": [],
   "source": [
    "from enum import Enum\n",
    "from typing import Union\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "import openai\n",
    "from openai import OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "6654b1e3-6c24-43d0-908b-6ef614f4aae4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:52.728513Z",
     "iopub.status.busy": "2024-10-01T03:05:52.727944Z",
     "iopub.status.idle": "2024-10-01T03:05:52.767202Z",
     "shell.execute_reply": "2024-10-01T03:05:52.765171Z",
     "shell.execute_reply.started": "2024-10-01T03:05:52.728465Z"
    }
   },
   "outputs": [],
   "source": [
    "class Table(str, Enum):\n",
    "    orders = \"orders\"\n",
    "    customers = \"customers\"\n",
    "    products = \"products\"\n",
    "\n",
    "class Column(str, Enum):\n",
    "    id = \"id\"\n",
    "    status = \"status\"\n",
    "    expected_delivery_date = \"expected_delivery_date\"\n",
    "    delivered_at = \"delivered_at\"\n",
    "    shipped_at = \"shipped_at\"\n",
    "    ordered_at = \"ordered_at\"\n",
    "    canceled_at = \"canceled_at\"\n",
    "\n",
    "class Operator(str, Enum):\n",
    "    eq = \"=\"\n",
    "    gt = \">\"\n",
    "    lt = \"<\"\n",
    "    le = \"<=\"\n",
    "    ge = \">=\"\n",
    "    ne = \"!=\"\n",
    "\n",
    "class OrderBy(str, Enum):\n",
    "    asc = \"asc\"\n",
    "    desc = \"desc\"\n",
    "\n",
    "class DynamicValue(BaseModel):\n",
    "    column_name: str\n",
    "\n",
    "class Condition(BaseModel):\n",
    "    column: str\n",
    "    operator: Operator\n",
    "    value: Union[str, int, DynamicValue]\n",
    "\n",
    "class Query(BaseModel):\n",
    "    table_name: Table\n",
    "    columns: list[Column]\n",
    "    conditions: list[Condition]\n",
    "    order_by: OrderBy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "7370999b-eff6-47c8-b603-241a3ed5d6b8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:05:54.567836Z",
     "iopub.status.busy": "2024-10-01T03:05:54.567268Z",
     "iopub.status.idle": "2024-10-01T03:05:56.806471Z",
     "shell.execute_reply": "2024-10-01T03:05:56.804657Z",
     "shell.execute_reply.started": "2024-10-01T03:05:54.567788Z"
    }
   },
   "outputs": [],
   "source": [
    "client = OpenAI()\n",
    "\n",
    "completion = client.beta.chat.completions.parse(\n",
    "    model=\"gpt-4o-2024-08-06\",\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"system\",\n",
    "            \"content\": \"You are a helpful assistant. The current date is August 6, 2024. You help users query for the data they are looking for by calling the query function.\",\n",
    "        },\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": \"look up all my orders in may of last year that were fulfilled but not delivered on time\",\n",
    "        },\n",
    "    ],\n",
    "    tools=[\n",
    "        # model as a tool\n",
    "        openai.pydantic_function_tool(Query),\n",
    "    ],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "e63dec58-68ee-48a6-b169-5f85f20e52d6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:01.769402Z",
     "iopub.status.busy": "2024-10-01T03:06:01.768789Z",
     "iopub.status.idle": "2024-10-01T03:06:01.815687Z",
     "shell.execute_reply": "2024-10-01T03:06:01.814113Z",
     "shell.execute_reply.started": "2024-10-01T03:06:01.769353Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">ParsedChatCompletion<span style=\"font-weight: bold\">[</span>NoneType<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'chatcmpl-ADNUx3Kglgje8egztzsq1BpFCxhA8'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">choices</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span>ParsedChoice<span style=\"font-weight: bold\">[</span>NoneType<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">finish_reason</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'tool_calls'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">index</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">logprobs</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">message</span>=<span style=\"color: #800080; text-decoration-color: #800080\">ParsedChatCompletionMessage</span><span style=\"font-weight: bold\">[</span>NoneType<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">refusal</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">role</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'assistant'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">function_call</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_calls</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">ParsedFunctionToolCall</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'call_1Kzkp6sUwGkXnjj8H2TV8c8h'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">function</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">ParsedFunction</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">arguments</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'{\"table_name\":\"orders\",\"columns\":[\"id\",\"status\",\"expected_delivery_date\",\"delivered_at\"],\"conditions\":[{\"column\":\"status\",\"operator\":\"=\",\"value\":\"fulfilled\"},{\"column\":\"ordered_at\",\"operator\":\"&lt;=\",\"value\":\"2023-05-31\"},{\"column\":\"ordered_at\",\"operator\":\"&gt;=\",\"value\":\"2023-05-01\"},{\"column\":\"expected_delivery_date\",\"operator\":\"&lt;\",\"value\":{\"column_name\":\"delivered_at\"}}],\"order_by\":\"asc\"}'</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'Query'</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">parsed_arguments</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Query</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">table_name</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;Table.orders: </span><span style=\"color: #008000; text-decoration-color: #008000\">'orders'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">columns</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000\">&lt;Column.id: </span><span style=\"color: #008000; text-decoration-color: #008000\">'id'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000\">&lt;Column.status: </span><span style=\"color: #008000; text-decoration-color: #008000\">'status'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000\">&lt;Column.expected_delivery_date: </span><span style=\"color: #008000; text-decoration-color: #008000\">'expected_delivery_date'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000\">&lt;Column.delivered_at: </span><span style=\"color: #008000; text-decoration-color: #008000\">'delivered_at'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">]</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">conditions</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Condition</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span><span style=\"color: #808000; text-decoration-color: #808000\">column</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'status'</span><span style=\"color: #000000; text-decoration-color: #000000\">, </span><span style=\"color: #808000; text-decoration-color: #808000\">operator</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;Operator.eq: </span><span style=\"color: #008000; text-decoration-color: #008000\">'='</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;, </span><span style=\"color: #808000; text-decoration-color: #808000\">value</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'fulfilled'</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Condition</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">column</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'ordered_at'</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">operator</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;Operator.le: </span><span style=\"color: #008000; text-decoration-color: #008000\">'&lt;='</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">value</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'2023-05-31'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Condition</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">column</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'ordered_at'</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">operator</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;Operator.ge: </span><span style=\"color: #008000; text-decoration-color: #008000\">'&gt;='</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">value</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'2023-05-01'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Condition</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">column</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'expected_delivery_date'</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">operator</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;Operator.lt: </span><span style=\"color: #008000; text-decoration-color: #008000\">'&lt;'</span><span style=\"color: #000000; text-decoration-color: #000000\">&gt;,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">value</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">DynamicValue</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">(</span><span style=\"color: #808000; text-decoration-color: #808000\">column_name</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #008000; text-decoration-color: #008000\">'delivered_at'</span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #000000; text-decoration-color: #000000; font-weight: bold\">]</span><span style=\"color: #000000; text-decoration-color: #000000\">,</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">order_by</span><span style=\"color: #000000; text-decoration-color: #000000\">=&lt;OrderBy.asc: </span><span style=\"color: #008000; text-decoration-color: #008000\">'asc'</span><span style=\"font-weight: bold\">&gt;</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">type</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'function'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">parsed</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">created</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1727751955</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">model</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'gpt-4o-2024-08-06'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">object</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'chat.completion'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">service_tier</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">system_fingerprint</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'fp_e5e4913e83'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">CompletionUsage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">completion_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">107</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">prompt_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">230</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">total_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">337</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">completion_tokens_details</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"font-weight: bold\">)</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "ParsedChatCompletion\u001b[1m[\u001b[0mNoneType\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'chatcmpl-ADNUx3Kglgje8egztzsq1BpFCxhA8'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mchoices\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0mParsedChoice\u001b[1m[\u001b[0mNoneType\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mfinish_reason\u001b[0m=\u001b[32m'tool_calls'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mindex\u001b[0m=\u001b[1;36m0\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mlogprobs\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mmessage\u001b[0m=\u001b[35mParsedChatCompletionMessage\u001b[0m\u001b[1m[\u001b[0mNoneType\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mrefusal\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mrole\u001b[0m=\u001b[32m'assistant'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mfunction_call\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mtool_calls\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1;35mParsedFunctionToolCall\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'call_1Kzkp6sUwGkXnjj8H2TV8c8h'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[33mfunction\u001b[0m=\u001b[1;35mParsedFunction\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33marguments\u001b[0m=\u001b[32m'\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"table_name\":\"orders\",\"columns\":\u001b[0m\u001b[32m[\u001b[0m\u001b[32m\"id\",\"status\",\"expected_delivery_date\",\"delivered_at\"\u001b[0m\u001b[32m]\u001b[0m\u001b[32m,\"conditions\":\u001b[0m\u001b[32m[\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"column\":\"status\",\"operator\":\"=\",\"value\":\"fulfilled\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"column\":\"ordered_at\",\"operator\":\"\u001b[0m\u001b[32m<\u001b[0m\u001b[32m=\",\"value\":\"2023-05-31\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"column\":\"ordered_at\",\"operator\":\">=\",\"value\":\"2023-05-01\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"column\":\"expected_delivery_date\",\"operator\":\"<\",\"value\":\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"column_name\":\"delivered_at\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m}\u001b[0m\u001b[32m]\u001b[0m\u001b[32m,\"order_by\":\"asc\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m'\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mname\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'Query'\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mparsed_arguments\u001b[0m\u001b[39m=\u001b[0m\u001b[1;35mQuery\u001b[0m\u001b[1;39m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[33mtable_name\u001b[0m\u001b[39m=<Table.orders: \u001b[0m\u001b[32m'orders'\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[33mcolumns\u001b[0m\u001b[39m=\u001b[0m\u001b[1;39m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[39m<Column.id: \u001b[0m\u001b[32m'id'\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[39m<Column.status: \u001b[0m\u001b[32m'status'\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[39m<Column.expected_delivery_date: \u001b[0m\u001b[32m'expected_delivery_date'\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[39m<Column.delivered_at: \u001b[0m\u001b[32m'delivered_at'\u001b[0m\u001b[39m>\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[1;39m]\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[33mconditions\u001b[0m\u001b[39m=\u001b[0m\u001b[1;39m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;35mCondition\u001b[0m\u001b[1;39m(\u001b[0m\u001b[33mcolumn\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'status'\u001b[0m\u001b[39m, \u001b[0m\u001b[33moperator\u001b[0m\u001b[39m=<Operator.eq: \u001b[0m\u001b[32m'='\u001b[0m\u001b[39m>, \u001b[0m\u001b[33mvalue\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'fulfilled'\u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;35mCondition\u001b[0m\u001b[1;39m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mcolumn\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'ordered_at'\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33moperator\u001b[0m\u001b[39m=<Operator.le: \u001b[0m\u001b[32m'<='\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mvalue\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'2023-05-31'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;35mCondition\u001b[0m\u001b[1;39m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mcolumn\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'ordered_at'\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33moperator\u001b[0m\u001b[39m=<Operator.ge: \u001b[0m\u001b[32m'>='\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mvalue\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'2023-05-01'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;35mCondition\u001b[0m\u001b[1;39m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mcolumn\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'expected_delivery_date'\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33moperator\u001b[0m\u001b[39m=<Operator.lt: \u001b[0m\u001b[32m'<'\u001b[0m\u001b[39m>,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   │   \u001b[0m\u001b[33mvalue\u001b[0m\u001b[39m=\u001b[0m\u001b[1;35mDynamicValue\u001b[0m\u001b[1;39m(\u001b[0m\u001b[33mcolumn_name\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'delivered_at'\u001b[0m\u001b[1;39m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   │   \u001b[0m\u001b[1;39m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[1;39m]\u001b[0m\u001b[39m,\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   │   \u001b[0m\u001b[33morder_by\u001b[0m\u001b[39m=<OrderBy.asc: \u001b[0m\u001b[32m'asc'\u001b[0m\u001b[1m>\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[33mtype\u001b[0m=\u001b[32m'function'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mparsed\u001b[0m=\u001b[3;35mNone\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mcreated\u001b[0m=\u001b[1;36m1727751955\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mmodel\u001b[0m=\u001b[32m'gpt-4o-2024-08-06'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mobject\u001b[0m=\u001b[32m'chat.completion'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mservice_tier\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33msystem_fingerprint\u001b[0m=\u001b[32m'fp_e5e4913e83'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33musage\u001b[0m=\u001b[1;35mCompletionUsage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mcompletion_tokens\u001b[0m=\u001b[1;36m107\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mprompt_tokens\u001b[0m=\u001b[1;36m230\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mtotal_tokens\u001b[0m=\u001b[1;36m337\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mcompletion_tokens_details\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m)\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from rich.pretty import pprint\n",
    "pprint(completion)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "02e075bc-c9cc-4aab-b662-413b11853105",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:05.443353Z",
     "iopub.status.busy": "2024-10-01T03:06:05.442768Z",
     "iopub.status.idle": "2024-10-01T03:06:05.454905Z",
     "shell.execute_reply": "2024-10-01T03:06:05.453299Z",
     "shell.execute_reply.started": "2024-10-01T03:06:05.443305Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Query(table_name=<Table.orders: 'orders'>, columns=[<Column.id: 'id'>, <Column.status: 'status'>, <Column.expected_delivery_date: 'expected_delivery_date'>, <Column.delivered_at: 'delivered_at'>], conditions=[Condition(column='status', operator=<Operator.eq: '='>, value='fulfilled'), Condition(column='ordered_at', operator=<Operator.le: '<='>, value='2023-05-31'), Condition(column='ordered_at', operator=<Operator.ge: '>='>, value='2023-05-01'), Condition(column='expected_delivery_date', operator=<Operator.lt: '<'>, value=DynamicValue(column_name='delivered_at'))], order_by=<OrderBy.asc: 'asc'>)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "completion.choices[0].message.tool_calls[0].function.parsed_arguments"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "088fd1cc-bdde-4016-9512-33ff812aa7a9",
   "metadata": {},
   "source": [
    "#### case 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "66079bda-489e-410f-9325-9453675889b3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:07.119593Z",
     "iopub.status.busy": "2024-10-01T03:06:07.119021Z",
     "iopub.status.idle": "2024-10-01T03:06:07.134148Z",
     "shell.execute_reply": "2024-10-01T03:06:07.132213Z",
     "shell.execute_reply.started": "2024-10-01T03:06:07.119544Z"
    }
   },
   "outputs": [],
   "source": [
    "class Step(BaseModel):\n",
    "    explanation: str\n",
    "    output: str\n",
    "\n",
    "class MathResponse(BaseModel):\n",
    "    steps: list[Step]\n",
    "    final_answer: str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "4f23a1d2-4877-402e-942e-3fbc66d982ba",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:08.429523Z",
     "iopub.status.busy": "2024-10-01T03:06:08.428949Z",
     "iopub.status.idle": "2024-10-01T03:06:11.076173Z",
     "shell.execute_reply": "2024-10-01T03:06:11.074308Z",
     "shell.execute_reply.started": "2024-10-01T03:06:08.429474Z"
    }
   },
   "outputs": [],
   "source": [
    "client = OpenAI()\n",
    "\n",
    "completion = client.beta.chat.completions.parse(\n",
    "    model=\"gpt-4o-2024-08-06\",\n",
    "    messages=[\n",
    "        {\"role\": \"system\", \"content\": \"You are a helpful math tutor.\"},\n",
    "        {\"role\": \"user\", \"content\": \"solve 8x + 31 = 2\"},\n",
    "    ],\n",
    "    response_format=MathResponse,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "6762252c-548d-460a-881c-e2ab6ee7fdac",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:11.080568Z",
     "iopub.status.busy": "2024-10-01T03:06:11.079988Z",
     "iopub.status.idle": "2024-10-01T03:06:11.114961Z",
     "shell.execute_reply": "2024-10-01T03:06:11.112863Z",
     "shell.execute_reply.started": "2024-10-01T03:06:11.080520Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">ParsedChatCompletion<span style=\"font-weight: bold\">[</span>MathResponse<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">id</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'chatcmpl-ADNVBpHPXi5RyEuWTjswIM09xw64M'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">choices</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span>ParsedChoice<span style=\"font-weight: bold\">[</span>MathResponse<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">finish_reason</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'stop'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">index</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">logprobs</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">message</span>=<span style=\"color: #800080; text-decoration-color: #800080\">ParsedChatCompletionMessage</span><span style=\"font-weight: bold\">[</span>MathResponse<span style=\"font-weight: bold\">](</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">content</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'{\"steps\":[{\"explanation\":\"First, we need to isolate the term with the variable on one side of the equation. We\\'ll start by subtracting 31 from both sides of the equation.\",\"output\":\"8x + 31 - 31 = 2 - 31\"},{\"explanation\":\"After subtracting 31 from both sides, we simplify the equation.\",\"output\":\"8x = -29\"},{\"explanation\":\"Next, to solve for x, we divide both sides of the equation by 8.\",\"output\":\"x = -29 / 8\"},{\"explanation\":\"Simplify the fraction to express the final result.\",\"output\":\"x = -3.625\"}],\"final_answer\":\"x = -3.625\"}'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">refusal</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">role</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'assistant'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">function_call</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">tool_calls</span>=<span style=\"font-weight: bold\">[]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">parsed</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">MathResponse</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">steps</span>=<span style=\"font-weight: bold\">[</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Step</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">explanation</span>=<span style=\"color: #008000; text-decoration-color: #008000\">\"First, we need to isolate the term with the variable on one side of the equation. We'll start by subtracting 31 from both sides of the equation.\"</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">output</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'8x + 31 - 31 = 2 - 31'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Step</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">explanation</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'After subtracting 31 from both sides, we simplify the equation.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">output</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'8x = -29'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Step</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">explanation</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Next, to solve for x, we divide both sides of the equation by 8.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">output</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'x = -29 / 8'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">Step</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">explanation</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'Simplify the fraction to express the final result.'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">output</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'x = -3.625'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">final_answer</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'x = -3.625'</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">]</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">created</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1727751969</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">model</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'gpt-4o-2024-08-06'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">object</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'chat.completion'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">service_tier</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">system_fingerprint</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'fp_e5e4913e83'</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"color: #808000; text-decoration-color: #808000\">usage</span>=<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">CompletionUsage</span><span style=\"font-weight: bold\">(</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">completion_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">145</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">prompt_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">127</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">total_tokens</span>=<span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">272</span>,\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   │   </span><span style=\"color: #808000; text-decoration-color: #808000\">completion_tokens_details</span>=<span style=\"font-weight: bold\">{</span><span style=\"color: #008000; text-decoration-color: #008000\">'reasoning_tokens'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">}</span>\n",
       "<span style=\"color: #7fbf7f; text-decoration-color: #7fbf7f\">│   </span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"font-weight: bold\">)</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "ParsedChatCompletion\u001b[1m[\u001b[0mMathResponse\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mid\u001b[0m=\u001b[32m'chatcmpl-ADNVBpHPXi5RyEuWTjswIM09xw64M'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mchoices\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0mParsedChoice\u001b[1m[\u001b[0mMathResponse\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mfinish_reason\u001b[0m=\u001b[32m'stop'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mindex\u001b[0m=\u001b[1;36m0\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mlogprobs\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[33mmessage\u001b[0m=\u001b[35mParsedChatCompletionMessage\u001b[0m\u001b[1m[\u001b[0mMathResponse\u001b[1m]\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mcontent\u001b[0m=\u001b[32m'\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"steps\":\u001b[0m\u001b[32m[\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"explanation\":\"First, we need to isolate the term with the variable on one side of the equation. We\\'ll start by subtracting 31 from both sides of the equation.\",\"output\":\"8x + 31 - 31 = 2 - 31\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"explanation\":\"After subtracting 31 from both sides, we simplify the equation.\",\"output\":\"8x = -29\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"explanation\":\"Next, to solve for x, we divide both sides of the equation by 8.\",\"output\":\"x = -29 / 8\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m,\u001b[0m\u001b[32m{\u001b[0m\u001b[32m\"explanation\":\"Simplify the fraction to express the final result.\",\"output\":\"x = -3.625\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m]\u001b[0m\u001b[32m,\"final_answer\":\"x = -3.625\"\u001b[0m\u001b[32m}\u001b[0m\u001b[32m'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mrefusal\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mrole\u001b[0m=\u001b[32m'assistant'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mfunction_call\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mtool_calls\u001b[0m=\u001b[1m[\u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[33mparsed\u001b[0m=\u001b[1;35mMathResponse\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[33msteps\u001b[0m=\u001b[1m[\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1;35mStep\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mexplanation\u001b[0m=\u001b[32m\"First\u001b[0m\u001b[32m, we need to isolate the term with the variable on one side of the equation. We'll start by subtracting 31 from both sides of the equation.\"\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33moutput\u001b[0m=\u001b[32m'8x + 31 - 31 = 2 - 31'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1;35mStep\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mexplanation\u001b[0m=\u001b[32m'After subtracting 31 from both sides, we simplify the equation.'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33moutput\u001b[0m=\u001b[32m'8x = -29'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1;35mStep\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mexplanation\u001b[0m=\u001b[32m'Next, to solve for x, we divide both sides of the equation by 8.'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33moutput\u001b[0m=\u001b[32m'x = -29 / 8'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1;35mStep\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33mexplanation\u001b[0m=\u001b[32m'Simplify the fraction to express the final result.'\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   │   │   \u001b[0m\u001b[33moutput\u001b[0m=\u001b[32m'x = -3.625'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   │   │   │   │   \u001b[0m\u001b[33mfinal_answer\u001b[0m=\u001b[32m'x = -3.625'\u001b[0m\n",
       "\u001b[2;32m│   │   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m]\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mcreated\u001b[0m=\u001b[1;36m1727751969\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mmodel\u001b[0m=\u001b[32m'gpt-4o-2024-08-06'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mobject\u001b[0m=\u001b[32m'chat.completion'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33mservice_tier\u001b[0m=\u001b[3;35mNone\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33msystem_fingerprint\u001b[0m=\u001b[32m'fp_e5e4913e83'\u001b[0m,\n",
       "\u001b[2;32m│   \u001b[0m\u001b[33musage\u001b[0m=\u001b[1;35mCompletionUsage\u001b[0m\u001b[1m(\u001b[0m\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mcompletion_tokens\u001b[0m=\u001b[1;36m145\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mprompt_tokens\u001b[0m=\u001b[1;36m127\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mtotal_tokens\u001b[0m=\u001b[1;36m272\u001b[0m,\n",
       "\u001b[2;32m│   │   \u001b[0m\u001b[33mcompletion_tokens_details\u001b[0m=\u001b[1m{\u001b[0m\u001b[32m'reasoning_tokens'\u001b[0m: \u001b[1;36m0\u001b[0m\u001b[1m}\u001b[0m\n",
       "\u001b[2;32m│   \u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m)\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pprint(completion)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "28ec93da-d375-44fb-a2b8-ad3755d1a0a4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-10-01T03:06:13.324698Z",
     "iopub.status.busy": "2024-10-01T03:06:13.324116Z",
     "iopub.status.idle": "2024-10-01T03:06:13.333570Z",
     "shell.execute_reply": "2024-10-01T03:06:13.331748Z",
     "shell.execute_reply.started": "2024-10-01T03:06:13.324650Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Step(explanation=\"First, we need to isolate the term with the variable on one side of the equation. We'll start by subtracting 31 from both sides of the equation.\", output='8x + 31 - 31 = 2 - 31'), Step(explanation='After subtracting 31 from both sides, we simplify the equation.', output='8x = -29'), Step(explanation='Next, to solve for x, we divide both sides of the equation by 8.', output='x = -29 / 8'), Step(explanation='Simplify the fraction to express the final result.', output='x = -3.625')]\n",
      "x = -3.625\n"
     ]
    }
   ],
   "source": [
    "message = completion.choices[0].message\n",
    "if message.parsed:\n",
    "    print(message.parsed.steps)\n",
    "    print(message.parsed.final_answer)\n",
    "else:\n",
    "    print(message.refusal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fb4d2e2-c42a-475a-86f9-f78f04542681",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "agent",
   "language": "python",
   "name": "agent"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
